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Abstract 


This  thesis  examines  the  transitive  closure  operation  and  more  general  linear  recursive  op¬ 
erations  in  deductive  databases  from  a  semigroup  standpoint.  An  algebraic  theory  capable 
of  completely  characterizing  all  redundance  encountered  upon  the  expansion  of  linear  re¬ 
cursive  inference  rules  is  first  developed,  and  then  the  scope  and  computational  complexity 
of  the  theory  is  studied.  In  addition,  we  sharpen  and  extend  earlier  results  on  efficient 
boundedness  testing  and  more  general  query  containment  problems. 
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Chapter  1 


Introduction 


This  thesis  is  an  examination  of  the  interplay  between  three  ideas:  relational  databases, 
transitive  closure  operations,  and  semigroups. 

1,1  Motivation 

Recently,  there  has  been  a  growing  realization  that  large  scale  relational  database  sys* 
terns  will  achieve  their  maximum  flexibility  and  usefulness  only  if  they  allow  sophisticated 
querying  operations.  Because  logic-based  methods  can  be  used  not  only  to  specify  the  sim¬ 
plest  querying  operations,  but  also  can  be  employed  as  full-fledged  programming  languages 
in  their  own  right,  much  attention  has  been  given  to  the  possibility  of  extending  the  more 
primitive  relational  quer3ring  operations  (e.g.,  select,  project,  join)  to  include  more  advanced 
constructs  from  logic  programming  languages,  such  as  Prolog  [ClocSlJ.  In  particular,  be¬ 
cause  first-order  relational  database  query  languages  lack  expressive  power  [AhU179],  the 
use  of  rtcursion  as  a  query  primitive  has  received  increasing  attention  in  the  database  com¬ 
munity  [ChHa85],  [GMN84],  [UUmSS].  Unfortunately,  the  addition  of  recursion  to  a  query 
language  carries  a  heavy  penalty  when  we  come  to  query  optimization,  because  it  has  been 
discovered  [Vard88],  [GMSV87]  that  many  natural  questions  one  would  like  answered  about 
general  recursive  programs,  even  in  the  more  simple  querying  contexts,  are  either  combi- 
natorially  difflcult  [SaYa80]  or  even  undecidable  [Vard88].  Recent  authors  have  therefore 
turned  to  the  identification  of  restricted  classes  of  recursive  programs  for  which  particular 
optimizations  can  be  performed  efficiently  [Ioan89],  [Sar89a],  [RSUV89],  and  much  useful 
effort  has  been  made  to  isolate  what  it  is  that  makes  a  given  program  difficult  to  optimize. 
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Given  a  databAse  query,  how  does  one  compute  the  answer  in  the  most  efhcient  wa}  . 
Unfortunately,  even  the  first  step  of  defining  what  we  might  mean  by  “the  most  efficient 
way"  is  problematic  [Ban86a].  Should  one  attempt  to  define  the  complexity  of  a  query  in 
terms  of  the  query  itself,  or  should  the  underlying  database  play  a  role  in  the  definition 
as  well?  Because  one  may  construct  examples  to  show  how  the  varying  structure  of  the 
underlying  database  can  exert  a  heavy  influence  not  only  on  the  complexity  of  alternative 
evaluation  algorithms,  but  also  on  the  site  of  the  resulting  query  answers,  one  it  tempted  to 
include  a  consideration  of  the  size  and  structure  of  the  database  in  the  definition  of  query 
complexity,  and  indeed  this  approach  has  its  advantages  [ChHa82].  However,  there  it  a 
parallel  investigative  line  that  has  also  been  taken.  We  may  simply  describe  it  as  syntactic 
query  analysis.  Here,  authors  have  attempted  to  divorce  the  analysis  of  a  query  from  its 
underlying  database,  and  they  have  shown  how  strong  results  about  the  complexity  of  eval¬ 
uating  a  recursive  query  may  be  won  by  considering  its  recursive  ezpanstons  in  conjunction 
with  the  idea  of  conjunctive  query  containment  In  their  strongest  possible  forms,  syntactic 
analysis  results  may  imply  either  that  a  recursively  defined  query  is  in  fact  replaceable  by  a 
nonrecursively  defined  one— a  boundedness  result— or  they  may  make  clear  some  aspect  of 
the  redundancy  inherent  in  recursive  expansion  of  queries  that  is  relevant  to  the  efficiency 
of  most  recursive  query  evaluation  algorithms  [Banc86b]. 

Here,  our  focus  will  be  restricted  to  the  simplest  manifestations  of  recursive  queries, 
the  linear  recursive  queries.  For  us,  such  queries  w^  manifest  themselves  as  pure  Bom 
clause  rules  without  function  symbols  or  negation;  their  syntax  will  conform  to  the  Dat- 
alog  language,  which  is  a  syntactic  subset  of  Prolog  more  fully  described  below.  At  the 
core  of  the  syntactic  analysis  of  such  queries  is  the  question:  given  a  set  of  linear  recur¬ 
sive  queries,  how  can  we  recognise  the  query  containments  between  their  various  recursive 
expansions?  Unfortunately,  the  general  problem  of  recognising  query  containment  is  NP- 
fChMeTTj,  so  a  retreat  from  the  search  for  efficient  and  all-encompassing  theories 
is  weU  advised.  Moreover,  even  some  of  the  simplest  possible  restricted  questions  about 
query  containments  between  general  recursive  query  expansions  have  been  remarkably  re¬ 
sistant  to  the  attacks  of  several  authors,  and  one  need  not  look  far  to  find  undecidability 
results  for  particular  problems. 

The  present  thesis  adds  to  the  growing  body  of  work  on  syntactic  query  analysis  by 
introducing  a  set  of  constructive  techniques  for  query  containment  problems  that  we  may 
loosely  describe  as  semigroup  methods.  At  the  center  is  the  observation  that  the  recursive 
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expansion  of  linear  recursive  rules  can  be  viewed  as  an  abstract  associative  operation  whose 
inverse  or  factomatian  properties  may  reveal  the  structure  of  query  containments.  Our 
methods  will  have  the  adstintage  that  when  they  work  completely,  they  will  give  us  a 
complete  and  succinct  description  of  all  the  query  containments  that  are  encotmtered  upon 
recursive  expansion  of  a  set  of  queries,  the  query  containment  information  will  come  to 
us  in  the  form  of  a  semigroup  p-  :,sentation.  Such  information  will  impinge  not  only  on 
the  often-studied  boundedness  properties  of  queries,  but  also  on  rule  commutativity  and 
recursive  redundance  properties  that  have  been  suggested  by  earlier  authors  in  syntactic 
query  analysis  contexts.  Our  results  will  lead  us  both  to  new  syntactic  analysis  problems  as 
well  as  to  reconsiderations  of  previously  proposed  issues,  and  we  shall  also  have  some  new 
things  to  say  about  some  of  the  simplest  manifestations  of  linear  recursive  queries. 


1.2  Databases  and  Datalog 

We  shall  take  Codd^s  relational  model  [CoddTO]  of  databases  as  our  starting  point.  The 
following  basic  definitions  foUow  those  to  be  found  for  example  in  [Sagi87]. 

We  shall  think  of  the  information  stored  in  a  database  as  represented  as  a  finite  number 
of  finite  mathematical  relations.  More  formallji  a  relation  Q  for  a  predicate  ^  is  a  set  of 
ground  atoms  of  some  fixed  arity.  Such  atoms  have  only  constants  and  no  variables  occurring 
in  them — ^for  example,  9(4,^*o/in,2,2)  might  be  a  typical  ground  atom.  A  coUection  of  such 
relations  (which  may  have  different  arities)  is  then  called  a  database.  If  are 

relations  for  predicates  9i  1 93,  • .  •  >  then  the  set  of  all  the  ground  atoms  that  occur  in  the 
various  relations  Qi  is  called  an  interpretation  or  a  structure. 

A  Datalog  program  is  a  finite  set  of  ffom  clause  rules  (see,  for  example,  [Ullm88]), 
each  of  which  is  required  to  have  only  predicates,  variables,  and  constants  in  its  head 
and  body.  Function  symbols,  negation,  as  well  as  other  common  constructs  from  logic 
programming  languages  (for  example,  lists,  Prolog  cuts,  or  arithmetic  operations)  are  not 
allowed.  The  Hom-clause  rules  defining  a  Datalog  program  are  often  more  simply  referred 
to  as  rules.  Each  rule  has  a  head,  appearing  on  the  left-hand  side  of  the  symbol :  and  a 
body,  appearing  on  the  right-hand  side  of  the  :  -  symbol.  The  head  of  a  rule  is  required  to 
be  a  single  afomte  formula  or  simply  atom,  that  is,  a  predicate  with  variables  in  each  of  its 
argument  positions.  The  body  of  a  rule  is  a  (possibly  empty)  conjunction  of  atoms.  A  rule 
with  an  empty  body  is  also  sometimes  called  a  fact 
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Suppose  n  is  a  fixed  Datalog  program.  If  a  predicate  p  occurs  as  the  head  predicate 
of  some  rule  of  II,  then  p  is  called  an  intensional  database  predicate,  (or  IDB  predicate). 
Predicates  ^  of  11  not  occurring  in  any  rule  head  are  called  extensional  database  predicates, 
(or  EDB  predicates).  The  input  to  a  Datalog  program  II  is  a  fixed  relation  Q  for  each 
extensional  predicate  j  of  H,  and  the  collection  of  all  such  relations  is  called  the  extensional 
database.  The  output  computed  by  11  is  in  effect  a  relation  for  each  intensional  predicate, 
and  it  is  called  the  intensional  database. 

Example  1.1  The  following  Datalog  program  11  has  two  rules,  ri  and  r2: 

ri  :  ancastor(XT)  :  -  fathar(XT) 

rj  :  anca8tor(XY)  :  -  ancaBtor(XA)  k  f athor(iT) 

Here,  the  predicate  father  is  extensional,  while  the  predicate  ancestor  is  intensional.  If 
we  additionally  specify  the  particular  input  extensional  database  relation 

father (Paul,  Paul  Jr.) 
fatherfPaul  Jr.,  Paul  III), 

then  the  output  computed  by  n  is  the  relation 

ancestor (Paul,  Paul  Jr.) 
ance8tor(Paul  Jr.,  Paul  III) 
ancestor (Paul,  Paul  III). 

I 

The  intuition  that  the  output  of  Datalog  program  should  be  definable  from  its  input 
by  starting  with  its  input  EDB  relations  and  then  applying  rules  exhaustively  to  derive 
all  output  IDB  reUtions  can  be  made  mathematically  precise,  and  in  fact  it  shall  be  our 
assumption  throi^out  that  the  method  ot  fixed  point  evaluation  [Ullm88]  is  used  to  assign  a 
unique  minimal  model  to  the  relations  of  a  Datalog  program,  given  its  input.  It  is  known  that 
the  minimal  model  containing  any  given  EDB  relations  is  unique,  and  that  it  corresponds 
exactly  to  the  set  of  facts  one  can  derive,  using  the  rules,  from  the  gpven  database.  We 
again  refer  the  reader  to  [Ullm88]  for  a  more  detailed  explanation  of  the  minimal  model 
construction. 


1.3.  LINEAR  RECURSION 


0 


1.3  Linear  recursion 

If  the  head  predicate  p  of  a  Datalog  rule  r  also  appears  in  its  body,  then  the  rule  r  is  said  to 
be  recursive.  If  r  is  recursive  with  head  predicate  p,  and  p  occurs  only  once  in  the  body  of  r, 
then  we  shall  say  that  r  is  a  rule  linear  recursive  in  p.  Regardless  of  whether  r  is  recursive  or 
not,  the  variables  of  r  that  occur  in  the  head  of  the  rule  are  called  its  distinguuhed  variables; 
other  variables  that  occur  only  in  the  body  of  r  are  called  nondistinguished  variables.  Except 
where  otherwise  noted,  we  shall  always  be  considering  Datalog  programs  consisting  of  one 
or  more  linear  recursive  rules,  with  each  linear  recursive  in  a  single  predicate  p. 

1.4  Semigroups 

A  semigroup  (5,  ■)  is  &  set  5  with  a  binary  operation,  denoted  by  •,  satisfying  the  associativity 
law.  z-{y-z)  =  (x-y)-z.  If5  additionally  has  an  identity  element  1  satisfying  z ■  1  =  l  -z  =  z 
for  all  elements  z  €  5,  then  5  is  called  a  monoid. 

In  this  thesis  we  shall  be  principally  concerned  with  semigroups  and  monoids  that  arise 
in  the  analysis  of  transitive  closure-like  rules.  Among  the  simplest  of  such  semigroups  are 
the  free  semigroups. 

1.4.1  Ftree  semigroups 

Let  A  be  a  set  called  an  alphabet,  the  elements  of  A  to  be  called  letters.  A  word  in  A 
is  a  nonempty  finite  sequence  ziZs-'-Zn  of  elements  of  A.  Two  words  ZiZs-'-Zn  and 
yilH  ’“Vm  ve  defined  to  be  equal  if  and  only  if  they  coincide  as  sequences — that  is,  we 
must  have  m  —  n  and  Xi  =  y.  for  1  <  t  <  n.  We  shall  use  the  notation  A*  to  denote  the  set 
of  all  words  in  the  alphabet  A.  If  we  define  a  binary  operation  •  on  A***  by  the  concatenation 
operation 

then  (A'*',  •}  is  a  semigroup  which  we  shall  call  the  free  semigroup  on  A.  The  implication 

ziZ2**'Xn  vs  yiys-'-Pm  =>  m  =  n  and  Zi  =  yi  for  1  <  t  <  n 

expresses  the  fact  that  every  word  w  €  A*^  has  a  unique  factorization  as  a  product  of 
elements  of  A. 
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A  svbsemtffrottp  T  of  »  semigroup  (5,  •)  is  a  subset  of  5  that  is  closed  under  •,  that  is, 
*i.*j  €  T  implies  XiXj  €  T.  If  A  is  a  subset  of  a  semigroup  5,  then  there  is  a  smallest 
subsemigroup  of  5  containing  .4,  which  we  shall  call  the  subsemiffnvp  generated  by  A,  and 
we  shall  use  the  notation  (A)  to  stand  for  this  semigroup.  The  semigroup  (A)  coincides  with 
the  set  of  all  finite  products  of  elements  of  A.  Any  subset  A  of  5  such  that  (A)  =  5  is  caUed 
a  set  of  generators  of  5;  there  always  exists  such  a  set,  for  example,  we  may  take  A  =  5.  If 
5  is  a  free  semigroup  on  the  alphabet  A,  then  (A)  s  A'*’  and  A  is  a  set  of  generators  of  5. 

A  homomorphism  xl>  from  a  semigroup  (S,  ■)  to  a  semigroup  (T,  o)  is  a  mapping  xl)  from 
S  into  T  such  that  t^(*  •  y)  =  xl>{x)  o  tl>(y)  for  all  x,  y  €  S.  The  following  elementar}’  result 
[Lall79]  connects  the  ideas  of  free  and  general  semigroups  via  the  use  of  homomorphisms: 

Theorem  1.2  Jfxl>isa  fixed  mapping  from  a  set  A  into  a  semigroup  S,  then  there  exists 
a  unique  homomorphism  from  the  free  semigroup  A*  into  S  such  that  t^(x)  si  for 
oil  X  ^  A.  Also,  xi)  is  onto  if  and  only  if  xj)(A)  is  a  set  of  generators  of  S. 

Proof  For  tu  =  x.x,  •••!„€  A+,  define  =  0(xi)t^>(x,)  •  •  •ti-(xn).  The  mapping  xi; 
is  therefore  forced  to  be  a  homomorphism,  and  we  see  in  fact  that  if  is  to  be  a  homomor¬ 
phism  at  all,  then  the  given  definition  it  forced  on  us.  So  it  unique.  Finally,  xl>{A)  will 
be  a  set  of  generators  of  5  if  and  only  if  each  s  =  V'(yi)t^(y2)  •  •  ‘t^(ym)  =  tfr(yiyj  •  "Vm)  for 
some  yi ,  •  •  • ,  ym  €  A.  Equivalently,  xjt  mutt  be  onto.  I 

1.4.2  Presentations 

Theorem  1.2,  when  applied  to  a  set  A  of  generators  of  a  semigroup  S,  yields  the  following 
corollary. 

Corollary  1.3  Every  semigroup  S  is  o  hemomorphie  image  of  the  free  semigroup  A^  on 
any  set  A  of  generators  of  S. 

The  corollary  it  important  because  it  lies  at  the  root  of  the  idea  of  a  semigroup  presen¬ 
tation,  which  it  it  our  next  task  to  describe. 

The  idea  of  a  semigroup  presentation  arose  most  importantly  in  the  context  of  certain 
decidability  problems  in  logic  [Thuel2],  (Post47].  Our  principal  interest  in  presentations 
will  be  to  use  them  to  capture  redundancy  occurring  amongst  the  recursive  expansions  of  a 
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database  query— but  here  we  anticipate  ourselves  a  bit.  For  now,  all  we  need  to  do  is  pick 
Up  the  basic  definitions. 

Let  5  be  a  semigroup.  A  set  A  is  called  a  set  of  generating  symbols  for  S  under  the 
mapping  :  A  -»  S  if  the  homomorphic  extension  xp  to  the  free  semigroup  (given  by 
defining  i/{xi  •  •  ■!„)  =  •  •  -^{xn)  »»  in  Theorem  1.2)  is  onto.  If  A  is  a  set  of  generating 

symbols  for  S  under  rp,  then  when  two  words  twi.tuj  €  A"^  satisfy  xp{wi)  =  v{w2),  we  say 
that  5  satisfies  the  relation  a>i  =  tuj,  or  that  ti»i  ss  trj  is  a  relation  in  S. 

If  we  are  given  a  set  of  relations  {toj  =  tuj,  i  €  /}  in  5,  then  we  define  a  word  t?  €  A 
to  be  directly  derivable  from  a  second  word  v'  €  A'*  if  either  v  =  rwiS  and  t»  =  ru\s,  or 
V  -  rtt's  and  v'  =  rwiS  for  some  t  €  /  and  r,j  €  A'*’.  Whea  v  =  t)o,vj,  -  =  v'  is  a 

sequence  of  words,  each  member  of  the  sequence  directly  derivable  from  its  predecessor  and 
successor  (if  they  exist),  then  we  say  that  v  is  derivable  from  v'  (in  n  steps). 

If  t»  is  derisable  from  v'  under  a  set  of  relations  {wi  =  wj,  i  €  /}  in  S,  then  v  =  v' 
is  itself  a  relation  in  S.  We  shall  say  therefore  that  v  =  u'  is  consequence  of  the  relations 
{lOi  =  tt'-,  i  €  /}  in  S,  or  sometimes  that  v  =  »'  is  implied  by  the  given  relations.  If  dl 
relations  in  5  are  consequences  of  {tui  =  tuj,  i  6  /},  then  (A  |  {»,•  =  tej,  i  €  /})  is  called  a 
presentation  of  S  defined  by  xp. 

Thus,  to  give  an  example  of  a  semigroup  5  it  is  enough  to  give  a  set  of  generators  A 
and  relations  =  tuj,  s  €  /}.  To  be  absolutely  precise,  we  should  say  that  the  semigroup 
5  is  in  fact  defined  to  be  the  quotient  of  A"*"  by  the  congruence  on  A*  generated  by  all 
pairs  {xvi,xvi),  with  t  €  I.  Here,  a  conymenceis  simply  an  equivalence  relation  p  on  S  that 
is  stable  under  left  and  right  multiplication,  and  the  elements  of  a  quotient  semigroup  are 
defined  to  be  the  equivalence  classes  of  a  congruence  with  multiplication  between  elements 
inherited  from  S.  We  shall  not  concern  ourselves  with  the  details  of  this  eongruence-quotietii 
construction  here  because  we  shall  carry  it  out  in  detail  for  our  particular  rule  expansion 
semigroups,  in  Chapter  2,  below;  for  precise  definitions  of  these  terms  and  a  verification 
congruence>quotient  construction  in  the  general  case,  consult  [Lall79]. 

1.4.3  'Eransformation  semigroups 

In  addition  to  semigroups  presented  by  generators  and  relations,  a  second  type  of  semigroup 
will  arise  frequently  in  our  work.  These  are  the  transformation  semigroups.  Here  we  are 
just  interested  in  the  basic  definitions. 

Suppose  that  n  is  a  positive  integer,  and  let  J*  w  {1, 2, . . . ,  n}.  A  transformation  of  In 
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is  just  a  function  f  ■  In  —*  In-  If  /  is  onto,  then  /  is  called  a  permutation.  The  set  of  all 
transformations  of  /„  admits  an  associative  multiplication  defined  by  function  composition, 
is  8  set  of  transformations  of  /„  closed  under  function  composition,  then  we  call 
F  a  transformation  semigroup. 

One  natural  way  to  represent  a  given  transformation  /  is  to  use  a  directed  substitution 
graph  on  the  vertices  7„,  with  a  directed  edge  i  —  j  H  f{i)  =  j.  We  see  immediately  that 
the  substitution  graph,  of  /  has  uniform  outdegree  1,  and  that  if  /  is  a  permutation,  then 
the  substitution  graph  of  /  partitions  into  directed  cycles,  some  of  which  may  be  trivial 
directed  cycles  (i.e.,  loop  edges)  corresponding  to  fixed  points  off.  The  substitution  graph 
of  a  general  transformation  f  may  have  vertices  belonging  to  no  directed  cycle,  although  if  i 
is  such  a  vertex,  then  there  is  a  minimal  /  >  1  such  that  /'(i)  is  contained  in  some  directed 
cycle  of  the  substitution  graph.  In  this  case,  the  vertex  i  is  called  stem,  and  I  is  called  its 
height.  Vertices  belonging  to  directed  cycles  of  the  substitution  graph  are  defined  to  have 
height  zero. 

For  example,  let  /  be  the  transformation  of  Iio  defined  by  the  mapping 

/i2345  6  789  10  \ 

^421  73  10  157  10  /' 

The  substitution  graph  of  /  has  three  directed  cycles.  Two  of  these  are  trivial  directed 
cycles  involving  the  fixed  points  2  and  10;  the  third  is  a  directed  cycle  l-*4-*7-*lon 
the  three  vertices  1,  4,  and  7  in  this  order.  The  remaining  vertices  3,  5,  6,  8,  and  9  are 
stem.  Of  these,  3,  6,  and  9  have  height  1;  the  vertex  5  has  height  2,  while  8  has  height  3. 

1.5  Overview 

In  this  section  we  survey  our  main  results  and  describe  the  thesis  organization.  Some 
techmcal  terms  are  necessarily  left  undefined  in  the  present  overview;  the  most  important 
terminology  appears  in  italics.  We  shall  also  relate  some  earlier  work  to  our  own. 

1.5.1  Thesis  organization  and  main  results 

Chapter  2  introduces  the  basic  mathematical  object  studied  in  this  thesis,  the  rule  expansion 
semigroup.  Roughly,  the  elements  of  a  rule  expansion  semigroup  consist  of  the  set  of  all 
top<down  recarsive  expansions  of  a  rule  set  S  of  linear  recursive  Datalog  rules,  every  such 
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rule  being  linear  recursive  in  a  single  IDB  relation  p.  The  mvJttplicatton  in  a  nile  expansion 
semigroup  is  defined  by  rule  expansion.  In  Section  2.2.3  we  introduce  a  simple  method  for 
the  calculation  of  rule  expansions  in  the  presence  of  multiple  recursive  rules,  and  call  the 
method  syntactic  expansion. 

There  is  a  natural  partial  ordering  >  by  query  containment  on  the  elements  of  rule 
expansion  semigroups;  most  problems  that  arise  in  syntactic  query  analysis  can  be  phrased 
in  terms  of  questions  about  this  partial  ordering.  An  important  elementary  property  of 
the  query  containment  ordering  that  is  exploited  throughout  the  thesis  is  that  recursive 
expansion  respects  query  containment,  a  proof  appears  in  our  “Splicing  Lemma,”  Lemma  2.4. 
We  close  our  introductory  chapter  with  a  few  simple  examples  of  rule  expansion  semigroups, 
and  give  a  brief  discussion  of  how  hosts  rules  fit  into  our  scheme. 

Detecting  query  containments  amongst  the  various  recursive  expansions  of  a  nile  set 
is  in  general  a  difficult  problem;  for  example,  a  recent  result  of  Saraiya  [Sar89b]  shows 
that  deciding  even  the  simplest  commutative  containment  relationship  rirj  v  r2rj  is  NP- 
complete  in  the  general  case.  However,  in  Chapter  3  we  show  how  it  it  possible  under 
certain  circumstances  to  win  a  complete  description  of  all  query  containments  encountered 
upon  recursive  expansion  of  a  set  of  rules  by  appealing  to  the  ideas  of  free  rides  and  rule 
factorization.  Basically,  the  idea  is  that  a  rule  is  free  if  it  satisfies  no  query  containments, 
and  if  the  elements  of  a  given  rule  set  can  be  written  as  recursive  expansions  of  free  rules, 
then  methods  from  free  semigroup  theory  may  be  used  to  give  a  complete  description  of  all 
query  containments  in  terms  of  a  semigroup  presentation.  Such  information  often  reveals 
information  about  the  rule  commutativity  or  boundedness  properties  of  recursive  queries, 
and  we  introduce  these  issues. 

In  Chapter  4  we  begin  a  study  of  the  applicability  of  our  techniques  in  the  general  case 
by  focusing  on  the  query  contaimnent  structure  of  a  single  linear  recursive  rule.  There  is 
a  close  connection  between  the  ideas  of  rule  freeness  and  simp  uniform  boundedness  here; 
in  particular,  a  recent  difficult  decidability  result  of  Vardi  [Vard90]  suggests  that  the  true 
complexity  of  our  techniques  may  be  “just  on  the  side  of  decidability.”  Our  own  contribution 
here  is  first  a  technical  containment  depth  result  that  gives  us  some  limited  information 
about  the  query  containment  structure  of  a  single  rule;  more  importantly,  the  notation  we 
introduce  will  allow  us  to  later  present  a  single  efficiently  testable  sufficient  condition  for 
sirup  unboundedness  that  simultaneously  subsumes  several  earlier  criteria  for  this  problem. 
There  is  a  close  connection  between  the  transformation  structure  of  the  substitution  graph  of 
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a  rule  and  the  EDB  subgoals  of  its  recursive  expansions  that  can  be  exploited  in  analyzing 
boundedness;  in  particular,  in  our  cut  and  cut  completion  conditions,  maximal  height  stem 
variables  play  a  critical  role. 

In  Chapter  5  we  turn  to  the  other  side  applicability  issues  when  we  take  up  the  problem 
of  rule  factorixation.  Intuitively,  a  linear  rule  r  factors  if  it  can  be  obtained  as  a  recursive 
expansion  of  simpler  rules  rj  and  rj.  Because  our  techniques  from  Chapter  3  assume  such 
factorizations  to  be  given,  the  factorization  problem  is  basic  to  the  applicability  of  our 
techniques.  We  begin  by  introducing  the  idea  of  an  irreducible  rule,  and  we  show  that  every 
rule  has  a  factorization  into  irreducibles.  Assuming  first  that  the  arity  of  a  linear  recursive 
rule  r  is  bounded,  we  give  an  algorithm  to  factor  r  in  poljmomial  time;  we  shall  call  our 
method  the  T-multigraph  algorithm.  When  the  arity  of  r  is  not  bounded,  the  r*multigraph 
method  fails  to  run  in  time  polynomial  in  the  size  of  the  nile,  so  we  develop  a  second  method 
called  the  flow  graph  algorithm  to  factor  general  rules  in  polynomial  time  as  well,  albeit  at 
a  sometimes  worse  time  complexity  than  that  in  the  bounded  arity  case.  Next,  we  consider 
unique  factorization,  and  we  show  that  although  unique  factorization  properties  are  usually 
not  present  in  rule  expansion  semigroups,  nevertheless  there  are  particular  situations  when  a 
weaker  notion  of  unique  factorizations  exists,  and  we  show  how  to  exploit  these  factorizations 
by  using  the  idea  graph  semigroups,  which  reappear  in  Chapter  6  as  well. 

Chapter  6  demonstrates  how  semigroup  techniques  may  be  brought  to  bear  on  the  study 
of  the  query  containments  of  linear  chain  rules,  which  are  a  simple  class  of  rules  that  arise 
in  both  practical  and  theoretical  settings.  The  thesis  closes  with  some  open  questions  and 
speculation  about  as  yet  tmproven  conjectures. 

1.5.2  Related  work 

The  work  most  similar  to  our  own  is  the  Ph.D.  thesis  of  JeiT  Naughton  [Naug87].  Most  of 
our  elementary  definitions  have  been  taken  directly  either  from  his  thesis,  or  alternatively 
from  a  sequence  of  papers  Naughton  wrote  in  coiyunction  with  Yehoshua  Sagiv  [Naug86b], 
{Naug86a],  (NaSa87],  (NaSa88],  {Naug88].  Most  importantly,  the  concepts  of  uniform  bound* 
edness  and  boundedness,  and  the  idea  of  a  chain  in  the  context  of  recursive  expansions  of 
linear  recursive  rules  (arising  for  us  in  Chapter  4)  can  be  traced  to  Naughton  and  Sagiv. 
Even  the  idea  of  a  free  rule  set,  which  is  central  to  the  present  work,  has  its  genesis  in 
an  observation  of  Naughton  that  contunment  freedom  between  recursive  expansions  is  “a 
useful  property.”  [Naug87].  Here,  we  have  taken  Naughton’s  comment,  and  have  expanded 
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it  into  another  thesis! 

Other  work  closely  related  to  our  own  can  be  found  in  the  papers  of  Yannis  loanni- 
dis  (Ioan85],  IIoan89],  [IoRa88].  In  particular,  interesting  discussions  of  rule  commutativity 
and  its  consequences  for  the  processing  of  linear  recursion  can  be  found  in  [ioan89|,  and  also 
in  (RSirV'89].  loannidis  and  Wong  (I0W088]  give  an  algebraic  account  of  linear  recursion  in 
relational  databases  that  in  part  anticipates  our  own  rule  expansion  semigroup  definition 
in  Chapter  2,  below.  With  a  few  exceptions,  all  the  semigroup  theory  we  employ  can  be 
found  in  the  clear  exposition  of  Lallonent  [Lall79]. 

A  recent  Ph.D.  thesis  by  Guozhu  Dong  (Dong88]  uses  language  quite  similar  to  our  own. 
Dong  writes  that  his  object  is  to  study  “the  composition  of  Datalog  mappings  in  order  to 
analyze  the  situation  where  a  sequence  of  Datalog  program  queries  are  es-aluated  serially," 
and  then  he  goes  on  to  study  the  “reverse  process  of  composition,"  which  he  calls  decom¬ 
position.  He  develops  the  notion  of  a  prime  program  as  one  which  cannot  be  decomposed, 
and  then  proves  results  relating  boundedness  to  the  decemposability  of  programs  into  single 
rule  primes.  In  its  general  spirit  and  outline,  Dong’s  work  bears  a  resemblance  to  our  own, 
in  particular  because  commutativity  and  unique  factorization  properties  arise  naturally  for 
him,  as  they  will  for  us.  The  difference  between  Dong’s  work  and  our  own,  however,  stems 
from  first  principles—while  our  notion  of  “composition”  corresponds  to  recursive  expansion 
of  rules,  Dong’s  composition  is  relative  to  cascading  execution  of  complete  Datalog  pro¬ 
grams.  For  example,  Dong  would  say  that  a  single  program  11  decomposes  into  single  rule 
programs  Hi  and  11]  if  the  relation  computed  by  II  it  obtainable  by  computing  the  fixed 
point  of  Hi  and  then  using  this  fixpoint  as  a  starting  point  for  a  second  II]  fixed  point 
computation,  while  we  would  say  that  II  decomposes  (actuaUy  factors,  in  our  language) 
if  n  it  the  rule  obtained  by  recursively  expanding  the  rule  Hi  by  the  rule  II].  The  two 
notions  do  not  appear  to  overlap — for  example,  it  it  not  even  known  whether  testing  for 
indecomposibility  in  Dong’s  sense  it  decidable,  while  our  rule  factorization  into  irreducibles 
is  decidable,  and  in  fact  can  be  carried  out  in  time  polynomial  in  the  size  of  the  rule. 

Still  more  discussion  of  other  authors’  work  and  its  rdation  to  our  own  will  be  found  at 
various  places  throughout  our  work. 


Chapter  2 


Rule  expansion  semigroups 


Our  basic  mathematical  object  of  study  is  the  rale  expansion  semigroup.  Roughly,  the 
oloTn^Titt  of  a  rule  expansion  semigroup  consist  of  the  set  ,7(5)  of  top-down  expansions  of 
a  set  5  of  linear  recursive  Datalog  rules,  with  “multiplication”  defined  by  rule  expansion. 
A  precise  definition  and  some  examples  appear  in  Section  2.2.4  below. 

Two  points  should  be  made.  First,  we  shall  consider  only  finite  sets  of  rules  5.  The 
restriction  is  a  natural  one:  real  programs  are  finite.  Second,  to  ensure  that  two  elements 
of  J(5)  can  always  be  “combined,”  (i.e.,  multiplied),  we  shaU  restrict  our  attention  only  to 
sets  5  l^nntaining  linear  recursive  rules  over  a  single  recursive  predicate  p.  Finally,  we  shall 
always  assume  the  head  predicate  of  such  a  recursive  rule  to  consist  of  distinct  variables. 
Even  with  our  restrictions,  we  shall  see  not  only  that  a  rule  expansion  semigroup  J{S) 
can  be  a  mathematical  object  of  considerable  complexity — for  example,  every  finite  group 
is  isomorphic  to  a  rule  expansion  semigroup— but  also  that  our  methods  will  apply  to  the 
study  of  more  simple  and  “natural”  Datalog  recursions,  such  as  the  simple  transitive  closure 
rule  for  graphs  (see  Section  2.1). 

Corresponding  to  every  rule  expansion  semigroup  ,7(5)  there  is  a  natural  partial  order¬ 
ing  ^  given  by  query  containment.  We  take  up  the  details  in  Section  2.2. 

Important  to  our  viewpoint  is  the  implicit  restriction  that  the  generating  set  5  of  a  rule 
expansion  semigroup  cont^  no  basis  (that  is,  nonrecursive)  rules.  Our  motivation  for  this 
is  twofold.  First,  by  removing  the  basis  rules  from  a  program,  we  endow  its 
recursive  rules  with  a  multiplicative  structure  that  can  be  studied  independently  from  its 
basis  rules.  Second,  often  bash  rules  “make  no  difference,"  in  the  sense  that  their  reintro- 
duction  into  a  program  often  does  not  change  the  query  containment  properties  that  are 


2.1.  LINEAR  RULES  AND  TOP-DOWN  EXPANSION 


13 


iiaportant  in  recursive  query  processing  applications:  for  example,  when  all  nonrecursive 
rule  bodies  contain  only  subgoals  that  appear  in  the  body  of  no  recursive  rule,  then 
it  can  be  shown  (see,  for  example,  [loanBo])  that  studying  the  query  containments  between 
expansions  of  the  recursive  rules  only  suffices  to  capture  all  query  containments.  In  Sec¬ 
tion  2.3  we  return  for  a  moment  to  consider  basis  rules,  and  introduce  questions  which  will 
be  studied  in  more  detail  later. 


2.1  Linear  rules  and  top-down  expansion 

A  natural  starting  point  is  the  simple  transitive  closure  rule  for  directed  graphs: 

ti  :  p(XY)  :  -  p(Xl)  t  e(lY).  (2.1) 

If  we  additionally  take  the  basis  rule 

6:p(XY)  «(XY), 

we  can  think  of  the  program  11  =  {u,6}  as  defining  a  “directed  path”  relation  p  from  a 
given  directed  “arc”  relation  e. 

Let  5  =  {«}.  Expanding  the  rule  tt  by  itself  recursively,  we  obtain  an  infinite  set  J{S) 
of  rules 

tt :  p(XY)  :  -  p(XA)  t  e(AY), 
tt»  :  p(XY)  :  -  p(Xl')  t  •(I'l)  »  .(AY), 

:  p(Xt)  :  -  P(XA")  »  «(A"A')  *  •(*'*)  *  «(AY), 

together  with  the  general  rule  for  every  t  >  3, 

tt*  :  p(XY)  p(XAt‘“^^)  t  k  •••  k  •(A'A)  k  •(AY).  (2.2) 

The  names  u.tt’.u®, ...  we  have  given  to  these  rules  are  meant  to  be  suggestive:  in  some 
sense  there  is  a  correspondence  between  this  set  J{S)  of  top-down  rule  expansions  and 
the  free  semigroup  {u,uu,uuu, . . .}  generated  by  the  formal  symbol  u  (with  “multiplication" 
corresponding  to  string  concatenation).  In  Example  2.7  below,  we  make  this  correspondence 
precise. 
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2.2  Partial  ordering  by  query  containment 

To  continue,  we  need  a  slightly  more  complicated  example.  Let  S  =  {a,t}  consist  of  the 
two  rules 

s  :  p(XY)  :  -  p(BA)  *  e(XB)  t  e(AY)  (2.3) 

t  :  p(XY)  :  -  p(AB)  t  e(BY)  k  e(YA)  t  •(XA),  (2.4) 

and  suppose  that  V  and  S  are  fixed  finite  sets  containing  initial  sets  of  p-  and  e-facts, 

respectively.  Then  we  can  use  the  two  rules  s  and  t  to  compute  new  j>-facts.  For  example, 

let 

V  =  {p(2,5).p(2,4)} 

be  an  initial  set  of  p-facts,  and  let 

£  =  {e(4,l),e(l,2),e(3,2),e(£,3).e(l,6),e(2,2)} 

be  an  initial  set  of  e-facts.  Then  by  using  the  rule  t  and  the  facts  p(2,5),  e(5,3),  e(3,2), 
and  e(2,2),  we  can  infer  thep-fact  p(2,3): 

t  :  p(2.3)  :  -  p(2,5)  k  e(S,3)  k  a(3,2)  k  .(2,2).  (2.5) 

Here,  we  have  used  the  bindings  X  =  2,  Y  =  3,  A  =  2,  and  B  =  5. 

More  facts  can  be  proved  by  recursive  expansion  of  the  rules  s  and  t.  For  example, 
substituting  the  rule  t  for  the  p- subgoal  in  the  rule  s,  we  obtain  the  rule 

si  :  p(XY)  :  -  p(A'B')  k  •(B'A)  k  «(AA')  k  •(BA')  k  •(IB)  k  •(AY).  (2.6) 

We  have  given  the  name  “si”  to  this  rule  in  order  to  sugg«.  ■  *hat  it  is  obtained  by  “multi¬ 
plying”  (i.e.  expanding)  the  rule  s  by  the  rule  t.  We  can  use  the  rule  si  to  infer  the  p-fact 
p(5,6):  we  have 

St  :  p(5,6)  :  -  p(2.4)  t  *(4,1)  k  •(1,2)  t  •(3.2)  k  •(5,3)  k  .(l.S). 

where  we  have  used  the  bindings  A  =  1,  A'  =  2,  B  =  3,  B'  w  4,  X  =  5,  and  Y  w  6.  Naively, 
we  may  expect  that  given  arbitrary  initial  fact  sets  V  and  £,  we  may  be  forced  to  expand 
the  two  rules  s  and  t  in  arbitrary  combinations  in  order  to  find  all  the  p-facts  implied  by 
these  two  rules.  However,  in  this  particular  case,  a  considerable  simplification  is  possible. 
In  fact,  any  p-fact  which  can  be  inferred  using  the  rules  s  and  t  together  can  be  inferred  by 
lining  the  rule  s  alone.  To  see  why,  we  need  to  develop  the  idea  of  a  containment  mapping. 
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2. 2,1  Containment  mappings 

We  r.  :urn  to  the  rules  s  and  t  above  (equations  and  2.4).  Let  4>  be  the  function  mapping 

the  set  of  head  and  body  predicates  of  s  into  ib‘.  set  of  head  and  body  predicates  of  t  as 
follows: 

0(p(XY))  =  pfXY) 

^(e(BA))  =  e(AB) 

^(e(XB))  =  e(Xl) 

<>(e(AY))  =  «(BY). 

The  mapping  ^  satisfies  the  following  four  properties: 

1.  <t>  maps  the  head  predicate  of  s  to  the  head  predicate  of  t,  and  these  predicates  have 
the  same  name. 

2.  0  maps  each  body  predicate  of  s  to  a  body  predicate  of  t  with  the  same  name. 

3.  If  gi  and  q,  are  predicates  of  a  with  identical  distinguished  variables  at  positions  i  and 

j,  respectively,  then  <^(91)  and  ^(jj)  have  identical  distinguished  variables  at  positions 
t  and  jf,  respectively. 

4.  llqi  and  q,  are  predicates  of  j  with  identical  nondistinguished  variables  at  positions 
i  and  i.  respectively,  then  ^(91)  and  <4(9,)  have  identical  variables  (distinguished  or 
nondistinguished)  at  positions  i  and  respectively, 

A  mapping  like  <l>,  satisfying  conditions  1-4  above,  is  caDed  a  containment  mapping. 

Corresponding  to  eve^  containment  mapping  ^  from  a  rule  to  a  rule  rj  there  is  a 
unique  derseei  mapping  ^  which  maps  the  set  of  variables  of  r,  into  the  set  of  variables 
of  rj,  and  agrees  with  ^  at  the  predicate  variable  level.  For  the  particular  containment 
mapping  ^  above,  the  derived  mapping  $  is  given  by 

m  =  X 

m  =  T 

0(i)  =  B 

^(B)  =  A. 


} 
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UTien  a  containment  mapping  0  :  ri  -»  rj  from  a  nJe  rj  to  a  rule  ej  exists,  then  any 
fact  that  can  be  concluded  by  using  one  application  of  the  rule  rj  can  also  be  concluded  by 
using  one  application  of  the  rule  rj.  (The  converse  is  also  true:  see  Theorem  2.1  below).  For 
example,  consider  the  derivation  2.5  above,  where  it  was  shown  how  to  obtain  the  p-fact 
p(2,3)  using  the  rule  t.  The  containment  mapping  ^  above  shows  us  how  to  derive  this 
same  fact  using  the  rule  s.  Here  are  the  details.  For  each  subgoal  in  the  body  of  j,  we  use 
the  derived  mapping  ^  and  t-derivation  of  p(2,3)  to  find  an  s-derivation  of  this  fact.  The 
{•derivation  of  the  p(2,3)  was  given  in  Equation  2.5,  and  had  bindings  X  =  2,  T  =  3, 1  =  2, 
and  B  =  5.  The  derived  mapping  ^  is  given  by 

^I)  =  X  =  2 
^T)  =  T  =  3 
^(1)  =  B  =  5 
^(B)  =  1  =  2 


The  associated  s*  derivation  is  then 

a  :  p($(X),$(T))  :  -  p(^B),^i))  k  •(^(X),«lB))  k  •(^A).^T)),  (2.7) 

or  equivalently, 

a;p(2,3)  p(2,5)  t  •(2,2)  »  *(5,3). 

To  show  the  nonexistence  of  a  containment  mapping  ^  from  a  rule  ri  to  a  rule  rj,  it  is 
often  convenient  to  work  with  derived  mappings.  For  example,  consider  the  two  top-down 
expansions  and  of  the  simple  transitive  closure  rule  2.1,  above: 

:  p(XT)  :  -  p(Xl')  k  •(I'l)  t  *(17), 

«•  :  p(XT)  :  -  p(U'0  *  •(iV)  t  •(I'l)  ft  •(IT). 

We  there  is  no  containment  mapping  “either  way“  between  these  two  rules.  Here,  we 
shall  show  only  that  there  is  no  containment  mapping  ^ :  u*  -*  ti*  (see  Example  3.3,  below, 
for  the  converse  in  a  generalized  form). 

Suppose  there  were  a  containment  mapping  ^  ti*.  Because  there  is  only  one 

occurrence  of  the  predicate  p  in  the  body  of  each  rule,  condition  2  on  ^  forces  ^p(Xi'))  = 
p(XA")-  The  derived  mapping  ^  therefore  has  ^(X)  =  X,  and  ^(1')  =  1".  Because  ^(1')  = 
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A",  we  must  have  ^(e(A'A))  =  e(i"A'),  since  the  latter  is  the  only  predicate  which  has 
a”  in  position  1.  The  latter  equation  in  turn  forces  <^(A)  ==  A'  in  the  derived  mapping. 
On  the  other  hand,  condition  3  forces  d(e(AY))  =s  e(lY),  because  •(AY)  is  the  only  body 
predicate  of  which  has  the  distinguished  variable  Y  occurring  in  position  2.  The  derived 
mapping  0  consequently  has  ^(A)  =  A,  which  together  with  i(A)  =  A',  above,  amounts  to 
a  contradiction:  0  is  not  single-valued.  We  conclude  that  no  such  containment  mapping  ^ 
exists. 

2.2.2  The  query  containment  theorem 

Now  let  j  and  t  be  arbitrary  Datalog  rules  with  the  same  head  predicate  p.  We  say  $ 
contains  t  (and  write  t  s)  if,  for  every  initial  database  D  over  the  predicates  appearing 
in  the  bodies  of  a  and  t,  the  set  of  p-facts  w^hich  can  be  derived  by  one  application  of  the 
rule  t  is  a  subset  of  the  set  of  p- facts  which  can  be  derived  by  one  application  of  the  rule  a. 
Note  that  we  do  not  consider  p- facts  that  arc  derivable  only  by  recursive  expansion  of  the 
rules  s  and  t  in  this  definition. 

The  following  theorem  can  be  traced  to  [ChMc77],  although  the  present  proof  follows 
the  argument  in  [Ullm88]  closely. 

Theorem  2.1  (The  query  containment  theorem)  Let  s  and  t  be  two  Datalog  rdes. 
Then  s  contains  t  if  and  only  if  there  exists  a  containment  mapping  ^  :  a  t. 

Proof  Suppose  first  that  there  is  a  containment  mapping  ^  :  a  ^  t,  and  let  p  =  p(xi  *  •  *  Zn) 
be  an  arbitrary  p-fact  which  is  t-derivable  over  a  particular  database  D.  We  need  to  show 
that  p  is  also  a*derivable  over  D.  We  proceed  as  we  did  with  the  a-derivation  2.7  of  the 
previous  section:  for  each  variable  v  (distinguished  or  nondistinguished)  occurring  in  a,  we 
bind  this  value  to  ^n).  Because  ^  is  a  function,  this  binding  scheme  is  consistent;  because 
^  maps  the  head  predicate  of  a  to  the  head  predicate  of  t,  the  p-fact  p  =  p(zx  •  •  *  Zn)  appears 
at  the  head  of  the  rule  a.  FinaUy,  each  resulting  bound  predicate  in  the  body  of  a  is  easily 
seen  to  be  a  fact  of  D.  Thus  the  effect  of  binding  each  variable  u  of  a  to  ^(v)  is  to  produce 
an  a*derivation  of  the  p-fact  p.  We  have  proven  the  theorem  in  one  direction. 

Conversely,  suppose  that  t  :<  a.  We  need  to  construct  a  containment  mapping  ^  :  s  ^  t. 
To  each  distinct  variable  v  occurring  in  t,  assign  a  unique  ground  symbol  r(t;).  Then  for 
each  predicate  q  in  the  body  of  t,  create  a  f-fact  by  substituting  t(v)  for  each  variable  v  in 
q.  Let  D  be  the  database  constructed  in  this  way.  If  the  head  predicate  of  t  is  p(Xi  •  •  Xn), 
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then  the  p-fact  p  =  p(7'(Xi)  •  •  •T(Xn))  is  t-derivable  over  D.  Because  t  <,  we  see  that 
p  is  also  a-derivable  over  P;  moreover,  the  facts  used  in  any  such  s-derivation  necessarily 
come  from  D.  We  now  have  in  hand  an  s*derivation  and  a  (-derivation  of  the  p-fact  p,  each 
over  the  database  D.  Here’s  how  to  construct  a  containment  mapping  ^  :  a  -»  (.  First,  we 
map  the  head  predicate  of  a  to  the  head  predicate  of  (,  as  we  must.  Then,  for  each  body 
predicate  9  in  a,  we  examine  the  corresponding  botmd  predicate  used  in  the  a-derivation  of 
p,  above.  This  boimd  predicate  is  necessarily  a  fact  of  D,  which  in  turn  arose  from  some 
body  predicate  r  of  t.  We  then  define  ^{q)  =  r.  (If  multiple  choices  for  r  exist,  the  choice 
may  be  made  arbitrarily).  That  ^  is  indeed  a  containment  mapping  follows  directly  from 
the  uniqueness  of  the  values  r(t;).  I 

2.2.3  Syntactic  expansion 

Up  to  this  point,  we  have  been  a  bit  imprecise  about  how  linear  Datalog  rules  are  re¬ 
cursively  expanded  to  form  new  rules  (such  as  the  rule  “st"  in  equation  2.6,  above).  In 
the  present  section,  we  shall  make  the  recursive  expansion  process  mathematically  pre¬ 
cise.  Our  method  is  basically  a  generalization  of  the  “ExpandRule”  procedure  described  by 
Naughton  [Naug86a],  although  our  “subgoal  substitutions”  are  perhaps  more  reminiscent 
of  the  “substitution  graphs”  of  [JAN87]  than  they  are  of  Naughton’s  “A/V  Graphs.”  In 
any  case,  where  both  [Naug86a]  and  [JAN87]  are  more  concerned  with  expanding  a  single 
recursive  rule,  here  we  formalize  the  process  by  which  an  arbitrary  set  of  linear  recursive 
rules  each  with  the  same  head  predicate  p  can  be  expanded  in  arbitrary  orders.  There  is  no 
mystery  in  our  method,  however:  it  is  little  more  than  a  formalization  of  how  one  carries 
out  recursive  expansion  by  hand.  We  shall  call  our  method  syntactic  expansion. 

To  do  an  expansion  of  a  linear  recursive  rule  u  by  a  second  sudx  rule  «  correctly,  we 
must  focus  on  two  issues.  First,  we  must  identify  the  way  in  which  some  variables  of  «  are 
passed  down  to  v  through  the  recursive  subgoal  p  of  u.  Second,  we  must  have  a  method  to 
distinguish  between  nondistinguished  variables  that  arise  newly  in  the  expansion  of  v  from 
their  counterparts  in  u.  For  example,  consider  the  two  rules  u  and  v  defined  as  follows: 

tt  :  p(XiXjX3X4)  :  -  p(XaXiAX4)  *  •(X3IB)  (2.8) 

t> :  p(XiX2X3X4)  :  -  p(AX,X3B)  A  1(X4X2CX3).  (2.9) 

To  expand  u  by  v,  we  first  form  subgoal  substitutions  corresponding  to  u  and  v.  The 
subgoal  substitution  corresponding  to  a  rule  is  obtained  by  positionally  pairing  each  variable 
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in  the  head  of  the  rule  with  the  corresponding  variable  in  the  recursive  p  lubgoal  in  the 
body  of  the  rule.  Thus  for  the  rule  u  we  have  the  subgoal  substitution 


(A'liA'j,  A'2iA'i,  A'3ii4,  A'^iA'^;^, 


(2.10) 


which  we  read  as  “In  tx,  JTi  is  replaced  by  A'j,  A'j  is  replaced  by  A'l,  A's  is  replaced  by  .4, 
and  Xa  is  replaced  by  X4.”  Similarly,  the  subgoal  substitution  for  v  is  written 

{Xx\A,  A'jiA'i,  Xi\Xi,  A'4,5},. 

Next,  we  must  compose  the  respective  subgoal  substitutions  to  obtain  a  third  substitu- 


(XilAlA',  A-2lX,!A'2,  A'jIA'jI^,  jrsjBIB')*.- 


(2.11) 


There  are  several  important  points  to  be  made.  First,  observe  that  to  form  the  sub¬ 
stitution  for  uv,  we  have  begun  with  the  substitution  for  v,  and  have  followed  it  with  the 
substitution  for  «  (not  the  reverse  order).  Also,  note  that  the  nondistinguished  variables 
A  and  B  of  v  have  been  replaced  by  the  new  formal  s3rmbols  A'  and  5',  respectively,  in 
the  u-half  of  the  uv  subgoal  substitution.  These  substitutions  are  necessary  in  order  to 
avoid  confusion  between  the  A’s  and  B's  of  u  and  those  of  v.  More  formally,  we  could  have 
written  the  symbols  (A|  A')^  and  as  part  of  the  u-subgoal  substitution  2.10,  above, 

to  stand  for  these  two  substitutions,  but  because  such  substitutions  are  inherent  in  recursive 
expansion  by  u  regardless  of  the  particiilar  rule  u,  we  call  them  implicit  substitutioTis,  and 
do  not  write  them  down  in  subgoal  substitution  expressions.  In  more  general  compositions 
of  subgoal  substitutions,  we  must  replace  A'  by  a  second  formal  symbol  A",  and  A"  by  A"', 
and  so  on,  to  that  in  general,  every  linear  recursive  rule  r  has  an  infinite  set  of  implicit 
substitutions  of  the  form  corresponding  to  every  variable  A  occurring  in  r 

that  doesn't  appear  in  the  head  of  the  rule. 

A  symbol  like  which  is  part  of  the  the  subgoal  substitution  expression  2.11 

above,  is  called  a  sisbstitMtion  chain,  and  we  read  it  as  “In  uv,  Xt  is  replaced  by  Xi,  which 
it  replaced  in  turn  by  A'}."  We  should  think  of  a  substitution  chain  as  no  more  than  a 
composition  of  mappings,  so  that  the  substitution  chain  (Aj|Ai|A}),„  is  really  just  a  more 
involved  way  of  writing  (Aj|Aj)„,.  However,  when  we  are  proving  things  about  recursive 
expansion,  it  sometimes  will  be  more  convenient  to  work  with  substitution  chains  in  their 
unsimpliiied  forms. 
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Now,  how  do  we  use  these  subgo&I  substitutions  to  expand  rules?  It  is  easiest  if  we  thinlc 
of  an  abstract  expansion  tree  to  which  the  appropriate  s'ariable  substitutions  are  applied  at 
every  node.  For  example,  to  expand  u  by  r,  we  first  write  do%-n  an  expansion  tree  of  the 
form 


P(AMA'2,  XilXuXM.  XM  <Xi,  A,  B) 


Note  that  to  the  EDB  subgoals  e  and  /,  we  have  applied  only  the  variable  substitutions 
corresponding  to  the  expansions  along  the  path  leading  from  these  subgoals  to  the  root  of 
the  tree,  so  that  e  has  no  subgoal  substitutions  applied  to  its  variables,  while  /  has  only  the 
u-substitution  applied  to  its  variables.  Next,  we  simplify  the  substitution  chains  at  every 
variable  position  in  every  node  of  the  tree  by  replacing  each  substitution  chain  with  its 
terminal  symbol,  obtaining  the  simplified  expansion  tree 


p{X^,Xx,Xi,X^) 


liXt,XuA,X;i  t(Xt,A,B) 


P(A',X2,X,B0 
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begin 

Write  down  the  rule  r  *  r.m. 
for  i  *  (m-1)  down  to  1  do  begin 

(1)  Apply  the  r.i  subgoal  substitution 
to  all  variables  in  the  body  of  r. 

(2)  Append  the  nonrecursive  subgoals  of  r^i  to  r, 

{m  Loop  invariant:  r  *  r.i  ...  r.m  ♦} 

end 

end 

Figure  2,1:  Deriving  r  =  rir2  -  r^hy  s}Txtactic  expansion. 

The  rule  expansion  uv  itself  can  now  be  read  off  from  the  root  and  leaves  of  the  tree: 
we  have 

uv  :  p(XiX2X3X4)  :  -  p(A'X2lB')  ft  1(X4XiC'A)  t  «(X3lB).  (2.12) 

Having  established  a  method  for  expanding  one  rule  ri  by  a  second  one  r:,  we  turn  next 
to  the  general  problem  of  syntactically  expanding  an  arbitrary  set  of  linear  recursive  rules 
in  a  specified  order.  Suppose  then  that  we  are  interested  in  expanding  rir2  -•  r„.  Because 
subgoal  substitutions  compose  “in  the  reverse  order,"  we  may  apply  the  simple  algorithm 
in  Figure  2.1. 

Example  2.2  As  an  illustration  of  syntactic  expansion,  suppose  we  take  the  three  rules 

:  p(XiX2X3)  :  -  p(X2XiA)  »  «(BX3) 
r,  :  p(XiX2X3)  :  -  p(AX,B)  k  1(X2CX3) 
ra  :  p(XiX2X3)  :  -  p(X2AXi)  k  g(X3). 

for  which  we  wish  to  form  the  rule  expaxision  ririTiti.  Following  the  algorithm,  we  begin 
by  writing  down  the  last  rule  of  the  desired  expansion,  r  =  rj. 

r  :  p(XiX2X3)  :  -  p(X2AXi)  k  g(X3). 

Entering  the  for  loop,  we  apply  the  rj  subgoal  substitution  (XilA''2,X2lXi,  to  all 

the  variables  in  the  body  of  r,  obtaining  the  working  expression 


r  :  p{XiX2l3)  :  -  p(XiA'l2)  *  g(A), 
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and  then  we  append  the  nonrecursive  subgoal  of  rj  to  r,  to  obtain  the  rule 
n’-3  :  P(XiX2X3)  :  -  p(Xii'X2)  ft  g(A)  ft  6(8X3). 

So  far,  we  have  passed  through  the  for  loop  once. 

On  the  next  pass  through  the  loop,  the  r2  subgoal  substitution  {XilA.XjlX'i.XaiR),^ 
is  applied  to  all  the  variables  of  r  =  tjrs,  and  then  the  nonrecursive  subgoal  f(X3CX3)  of  r2 
is  appended  to  r.  We  obtain  the  nile 

rjrira  :  p(XiX2X3)  :  -  p{AA"X:)  ft  g(A')  ft  e(B'B)  ft  f{X2CX3). 

There  is  only  one  more  pass  through  the  loop  to  be  completed,  corresponding  to  the  rj 
occurrence  at  the  beginning  of  desired  expansion  rir2rir3.  Passing  through  steps  (1)  and 

(2)  of  the  syntactic  expansion  algorithm  a  final  time,  the  desired  rule  expansion  is  seen  to 
be 

nrsni-a  :  p(XiX2X3)  :  -  p(A'a'"22)  ft  g{A")  ft  e(B"B')  ft  f(XiC'A)  ft  «(BX3). 


2.2.4  The  semigroup  *7(5) 

Let  5  be  an  arbitrary  nonempty  finite  set  of  linear  recursive  Datalog  rules,  each  with 
the  same  head  predicate  p,  and  let  j(S)  be  the  (infinite)  set  of  aU  finite-depth  top-down 
syntactic  expansions  of  the  rules  of  S.  If  we  give  names  uj, . . . ,  ttn  to  the  rules  in  5,  then 
we  can  think  of  the  elements  of  .7(5)  as  in  one-to-one  correspondence  with  the  set  U‘  of 
all  finite  words  in  the  alphabet  U  =  {ui, . . When  two  top-down  expansions  a  and  t 
in  j(S)  simultaneously  stand  in  the  relations  a  ^  f  and  t  ^  a,  then  we  write  a  :s  t,  and  say 
"a  is  equivalent  to  t." 

Lemma  2.3  The  relation  2:  defines  an  equivalence  relation  on  .7(5). 

Proof  To  see  that  2:  is  reflexive,  it  suffices  to  note  that  we  always  have  the  trivial 
containment  u  <  u  (or  any  rule  u.  Also,  by  definition  2:  is  symmetric.  To  show  that  a 
is  transitive,  suppose  that  we  have  three  rules  a,  t,  and  u,  and  that  a  ^  t  and  t  &  «.  We 
want  to  show  that  a  2;  u.  Let  ^  :  a  (  and  ^  :  t  — *  u  be  containment  mappings. 
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Then  we  claim  that  the  composite  mapping  ^  =  4>24>i  defines  a  thiid  containment  mapping 
:  a  — •  u.  The  verifications  of  the  four  necessary  conditions  on  ^  are  easily  checked.  First, 

^  must  map  the  head  predicate  of  s  to  the  head  predicate  of  u,  since  <i>i  maps  the  head 
predicate  of  s  to  the  head  predicate  of  t,  and  ^  maps  the  head  predicate  of  t  to  the  head 
predicate  of  u.  Similarly,  <t>  must  map  each  body  predicate  of  a  to  a  body  predicate  of  u  with 
the  same  name,  since  (f>i  and  <h  individually  have  this  property.  FinaUy,  if  ?i  and  jj  are 
predicates  of  a  with  identical  (distinguished/nondistinguished)  variables  at  positions  i  and 
j,  respectively,  then  ^(?i)  and  ^(92)  must  also  have  identical  (distinguished/distinguished 
or  nondistinguished)  variables  at  positions  t  and  ;,  respectively,  again  precisely  because 
and  ^2  individually  have  these  properties. 

We  have  shown  therefore  that  a  :<  t  and  t  :<  u  together  imply  a  i  u.  Since  the  proof  of 
the  opposite  containment  is  completely  symmetric,  we  omit  it,  and  conclude  that  a  ^  u,  as 
required. 

I 

W’e  are  now  ready  to  define  the  idea  of  a  ride  expansion  setnignup  S{S).  The  elements 
of  S{S)  are  the  equivalence  classes  of  the  relation  2:  on  j{S).  The  “multiplication”  in 
J{S)  is  defined  by  rule  expansion:  to  form  the  product  Jj  J2  of  two  equivalence  classes 
Ji,Ji  €  J(5),  one  first  takes  in  j(S)  two  representative  rule  expansions  ji  €  Ji  and 
is  €  Ji-  After  finding  the  equivalence  class  J  €  J{S)  to  which  the  rule  j  =  j\ji  in  j{S) 
belongs,  one  finally  defines  J\Ji  =  J  in  J{S). 

There  are  several  points  to  be  checked.  First,  we  must  check  that  this  process  is  well 
defined;  i.e.,  we  must  verify  that  the  equivalence  class  to  which  j  belongs  is  independent  of 
the  choice  of  the  representatives  ji  and  jj. 

Lemma  2.4  (The  splicing  lemma)  Let  a,  t,  u,  and  v  be  four  linear  recursive  Datalog 
rules  each  with  the  same  head  predicate  p.  Suppose  that  sxii  and  «  2:  ».  Then  su  2:  tv. 

Proof  In  light  of  the  query  containment  theorem  (Theorem  2.1),  it  is  possible  to 
prove  the  splicing  lemma  by  either  of  two  methods.  One  can  either  work  nonconstructively 
with  the  abstract  containment  and  monotonicity  properties  of  Datalog  rules  (see  below),  or 
alternatively  one  can  produce  a  constructive  proof  by  working  with  containment  mappings 
and  the  syntactic  e2cpansion  of  rules.  It  turns  out  that  the  former  argument  is  simpler, 
and  we  shall  present  it  below.  But  because  the  constructive  proof  yields  some  additional 
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information  about  the  structure  of  containment  mappings  ^  :  su  —  (v  that  we  shall  exploit 
in  Chapter  4,  we  shall  present  the  latter  argument  first. 

By  assumption,  there  are  containment  mappings  (t>i  :  t  —  s  and  62  ■  v  —  u.  \Vc  shall 
show  that  these  two  mappings  can  be  “spliced  together**  to  obtain  a  third  containment 
mapping  (f>  :  tv  -*  su.  In  other  words,  we  wiU  have  shown  that  the  conditions  s  ^  t  and 
u  ■<v  together  imply  »u  ^  tv.  Because  the  proof  of  the  opposite  containment  is  completely 
symmetric,  we  omit  it. 

Without  loss  of  generality,  we  shall  assume  that  the  linear  recursive  predicate  p  has  arity 
a  >  1,  and  that  the  head  predicates  of  t,  s,  u,  and  v  are  all  p{XiX2  -  ’-Xa). 

In  the  figure  below,  we  illustrate  the  two  top-down  expansions  »u  and  tv.  As  indicated, 
the  rule  su  (respectively,  tv)  is  obtained  from  s  (respectively,  t)  by  expanding  its  p-subgoal 
by  the  rule  u  (respectively,  v).  We  have  used  triangles  to  indicate  the  collection  of  non¬ 
recursive  predicates  in  a  given  rule  expansion,  and  we  have  omitted  the  variables  of  the 
p-predicates  for  clarity. 


su 


tv 


P  P 


The  boxes  we  have  put  around  the  lower  halves  of  each  rule  expansion  are  meant  to 
draw  particular  attention  to  the  following  point,  which  follows  directly  £rom  the  definition 
of  recursive  expansion:  what  appears  in  the  box  in  a«  is  precisely  the  rule  «,  except  that 
the  a-subgoal  substitution  has  been  applied  to  all  of  its  variables,  while  what  appears  in  the 
box  in  tv  is  precisely  the  rule  v,  except  that  the  t-subgoal  substitution  has  been  applied  to 
all  of  its  variables. 
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Also,  because  61  is  a  contaiiunent  mapping,  the  following  two  properties  are  also  imme¬ 
diate: 

I.  If(A'i;X^),,then  {Xi'.Xj),. 
n.  If  (A\i.4)„  {A'^1.4),,  and  {Xi\V)„  then  (A'^IV),. 

In  property  II.,  the  variable  A  is  an  arbitrary  nondistinguished  variable,  while  V  is 
allowed  to  be  any  variable  (distingtiished  or  nondistinguished). 

In  order  to  construct  explicitly  a  containment  mapping  <(>  :  tv  -*  sxi,  the  containment 
mappings  <t>i  and  ^  may  be  spliced  together  in  th.-  following  way:  we  make  0  agree  with 
on  the  nonrecursive  t-subgoals  in  tv,  and  we  make  4>  agree  with  ^  on  the  p-subgoal  and  on 
the  nonrecuTsive  v-subgoals  of  tv.  In  other  words,  ^  maps  the  tw-predicates  in  triangle  1  of 
the  figure  to  the  su-predicates  of  triangle  3  in  the  figure  exactly  as  maps  these  predicates 
from  f  to  3;  also  ^  maps  the  predicates  in  triangle  2  to  the  predicates  in  triangle  4  exactly 
as  th  maps  these  predicates  from  v  to  u.  Finally,  (ft  maps  the  terminal  p-predicate  at  the 
lower  left-hand  comer  of  tv  to  the  corresponding  p-predicate  at  the  lower  left-hand  comer 
of  au.  Of  course,  ^  is  also  made  to  map  the  head  predicate  of  tv  to  the  head  predicate  of 
au. 

We  Haim  that  the  mapping  ^  so  constmeted  is  a  containment  mapping  from  tv  to  au. 
We  shall  check  each  of  the  four  containment  axioms  in  turn. 

Note  first  that  by  definition,  ^  maps  the  head  predicate  of  tv  to  the  head  predicate  of 
3U,  and  these  predicates  have  the  same  name.  So  we  have  verified  condition  1  on  Second, 
tf)  must  map  the  body  tw-predicates  to  body  att-predicates  of  the  same  name,  because 
and  h  *1®  *®  individually.  We  have  therefore  verified  condition  2  on  t(>.  Only  conditions  3 
and  4  remain  to  be  checked. 

We  first  turn  to  the  simpler  condition  3.  Because  we  have  assumed  that  each  of  the 
four  rules  a,  t,  u  and  v  has  head  predicate  p( ATi  JTj  •  •  •  Jfa),  verifying  condition  3  on  ^  comes 
djwn  to  checking  that  if  Xi  is  any  distinguished  variable  in  tw,  then  ^(X,)  =  A'i  in  au. 
The  verification  is  trivial  for  distinguished  variables  that  occur  in  triangle  1,  because  h  is 
a  containment  mapping  and  no  subgoal  substitutions  are  applied  to  any  of  these  variables 
when  t  is  expanded  by  w.  Also,  because  ^  is  a  contmnment  mapping  and  we  have  property 
I.,  above,  the  verification  is  also  immediate  for  distinguished  variables  of  tw  that  occur  either 
in  triangle  2  or  in  the  recursive  p-subgoal  of  tw. 
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o  veri/y  condition  4  on  d,  snppose  that  „  and  „  predicates  or  Iv  tcith  the  same 
noni.tn,p..sh.d  friable  appearing  in  position.  (  and  j.  respectirel,.  We  mnst  verify  that 
(9i)  an  <^(92)  have  identical  variables  (distinguished  or  nondistinguished)  at  positions 
»  an  j,  respectively,  in  su.  The  only  case  to  be  considered  which  is  not  an  immediate 
consequence  of  property  U  and  the  fact  that  and  6,  are  containment  mappings  occurs 
w  en  is  a  predicate  of  triangle  1,  and  ft  is  either  the  recursive  p  subgoal  or  a  predicate  of 
triangle  2.  In  this  case,  ft  and  ft  share  some  nondistinguished  variable  A  that  is  necessarily 

P^sed  through  the  intermediate p-subgoal  ottv,  and  we  must  verify  that  ^  define,  the  s^lue 
<P[A}  uniquely. 

^  Suppose  then  that  is  some  \-ariable  V  in  the  rule  a.  We  claim  that  that  (^(A)  =  I* 

In  proof,  note  that  in  order  for  the  variable  A  to  passed  to  the  expansion  by  t  in  tv, 
e  et  must  have  a  subgoal  substitution  of  the  form  {A'*|44),  for  some  ib,  where  A'* 
also  appears  at  position  j  in  ft  (in  v).  Also,  because  ^A)  =  V  and  is  a  containment 
mapping,  the  rule  a  has  the  subgoal  substitution  (A'a|V*)..  Now  because  ^  is  a  containment 
mapping,  ti  must  have  the  variable  .V*  at  position  j  in  *(ft).  and  when  a  is  expanded  by 
u,  the  last  A'fc  is  replaced  by  V,  so  that  ^(A)  =  V,  as  we  claimed. 

Therefore  au  :<  tv,  as  required. 


The  spbcmg  Icnima  cm  be  proved  noocoostroctively  by  referring  to  tbe  foBowing  mono- 
fonscsl,  property  of  Datnlog  mies:  i/firin,  .  Da^  nd.  ,  cn«r  .  delebss.  D  . 

..t  ./  on^er.  A,  end  fj-  i.  .  deleloe,  contefnisHf  C,  IA«,  r  « .« 
contosnsn,  4  srtes.  ,  i,  yired  com-  jy.  The  monotonicity  property  for  D.t  Jog  ndes  foUons 
from  the  monotonicity  propertie.  of  the  bmic  relntionj  Jgebr.  operntion,  onion,  select, 
project,  and  product,  and  a  proof  can  be  found  in  (UUmSS). 

With  the  m  aotonicity  property  in  hand,  we  now  can  prove  the  splicing  lemmaas  follows. 
Let  2?  be  a  database  and  suppose  that  a  V  t  and  u  V  t.  If  po  i,  *  p-fact  returned  by  the 
e  tv  over  D,  then  every  intermediate  p-fact  pi  at  the  head  of  v  in  a  tv  proof  tree  for  po 
can  also  be  derived  at  the  head  of  u  in  tome  au  proof  tree.  Therefore,  whatever  p-facts  P„  j, 
CM  reach  the  p-subgoal  of  I  through  t  in  tv  proof  trees  can  also  reach  the  jvsubgoal  of  a 
t  ough  tt  m  au  prood  trees,  and  these  respective  databases  of  p-facts  P,j,  and  stand 
m  the  relation  C  Now  by  monotonicity  and  the  fact  that  a  v  t,  we  conclude 
that  PO  is  also  a  p-fact  returned  by  the  rule  au  over  Z>.  So  au  v  tv,  as  required. 
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Recall  that  a  semi^vp  is  a  nonempty  set  Q  together  with  an  associative  binary  op¬ 
eration  on  G  [Hu74].  To  verify  that  the  set  J{S),  together  with  the  binary  operation  of 
syntactic  expansion,  is  indeed  a  semigroup,  we  must  check  that  the  operation  of  top-down 
recursive  expansion  of  linear  Datdog  rules  is  associative]  i.e.,  for  any  three  rules  rj,  r2,  and 
fa,  the  rule  (rir2)r3  obtained  by  expanding  ri  by  r2,  and  then  expanding  the  result  by  r^,  is 
equivalent  to  the  rule  fi(r2»‘3)  obtained  by  expanding  fi  by  the  rule  obtained  by  expanding 
T’j  by  ra: 

Lemma  2.5  Suppose  ri,  r^,  and  ra  are  linear  recursive  Datalog  rules  each  with  the  same 
head  predicate  p.  Then  (rjr2)ra  c:  ri(r2ra). 

Proof  It  suffices  to  check  that  the  operation  of  subgoal  substitution  composition  is  itself 
an  associative  operation.  To  do  this,  suppose  that  {U\V}^_  ,  {T\V)^^,  and  (517),^  are  three 
substitutions  of  ri,  ra  and  ra,  respectively.  Then  (5iriC7|l'’),|,j,j  =  regardless  of 

whether  we  think  of  the  composition  as  accomplished  according  to  the  “(rir2)ra  grouping” 
Sl(r|t7|l0.  or  alternatively  according  to  the  “ri(r2ra)  grouping"  (S|r|I7)lX',  so  that  the 
variable  patterns  in  the  subgoals  of  rjrara  do  not  depend  on  the  order  in  which  we  expand 
these  three  rules.  I 

Summing  up  these  results,  we  can  say  that  while  the  relation  v  on  the  set  S{S)  of 
recursive  expansions  of  5  is  a  quasiorder  (i.e.,  reflexive  and  transitive),  J{S)  itself  is  a 
partially  ordered  semigroup  with  order  relation  >. 

Here  are  ?ome  simple  examples  of  rule  expansion  semigroups. 

Example  2.6  Let  5  =  {a}  consist  of  the  single  rule 

s  :  p(XT)  :  -  p(YX).  (2.13) 

The  set  j{S)  =  {a, a*, a*,...}  consists  of  all  top-down  expansions  of  the  rule  a,  and  is 
infinite.  However,  it  is  easy  to  see  that  we  have  a*'  2:  a*'  and  a**~*  c:  a*^"*  every  pair 
of  positive  integers  t  and  j,  so  that  j{S)  partitions  into  exactly  two  equivalences  classes, 
namely 

Ji  =  {3*‘  1  i  >  1}, 

and 

J2  =  {a«-Mi>l}. 
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The  rule  expansion  semigroup  J(S)  =  {Ji,J2}  is  isomorphic  to  the  additive  group  Z2  of 
integers  taken  modulo  2.  I 

Example  2.7  Let  5  =  {u}  consist  of  the  transitive  closure  rule  2.1  of  Section  2.1,  above: 

u  :  p(XY)  :  -  p(Xl)  t  e(AY). 

In  this  example,  the  relation  i  on  the  set  j{S)  =  {u,  u*,  u®, . . .}  of  all  top-down  expan¬ 
sions  of  the  rule  u  is  trivial:  every  element  of  j(S)  defines  its  own  singleton  equivalence 
class.  (For  a  proof,  see  Example  3.3  below).  Thus,  we  have  a  one-to-on*  'orrespondence 
between  j(S)  and  JiS),  and  we  can  identify  every  equivalence  class  J,-  €  J(S)  with  the 
unique  rule  expansion  u*  which  defines  it.  The  rule  expansion  cemigroup  S(S)  is  therefore 
isomorphic  to  the  free  semigroup  U*  =  {u,  uu,  uuu, . . .}  generated  by  the  formal  sjTnbol  u. 


Example  2.8  Let  5  =  {u,  v}  consist  of  the  two  rules 

u  :  p(XY)  :  -  p(XA)  t  e(AY) 

V  :  p(XY)  :  -  p(AY)  A  e(XA). 

The  elements  of  S(S)  are  in  one-to-one  correspondence  with  the  set  U'  of  all  finite  words 
over  the  formal  alphabet  U  =  {u,  v}.  It’s  not  hard  to  check,  however,  that  we  have  uv  2:  vu. 
We  have 

uv  :  p(XY)  :  -  p(A'A)  k  e(XA0  k  e(AY) 


«ii :  p(XY)  :  -  p(AA')  k  •(A'Y)  k  •{Ik), 

and  we  can  show  there  are  contiunment  mappings  both  ways  between  these  two  rules.  In 
fact,  the  function  ^:uv  vu  given  by 

^(p(XY))  =  p(XY) 

^(p(i'A))  =  p(AA') 

^(e(XA'))  =  aClA) 

^(e(lY))  =  «(A'Y). 
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is  a  containment  mapping  whose  inverse  not  only  exists,  but  also  is  a  second  containment 
mapping  :  vu  -*  uv.  More  generally,  one  finds  that  two  top-down  expansions  Jj  and 
Jj  in  j{S)  stand  in  the  relation  Ji  i  Jj  if  and  only  if  they  involve  the  same  number 
of  expansions  by  the  rules  u  and  v  individually.  The  rule  expansion  semigroup  J{S)  is 
therefore  isomorphic  to  the  free  commutative  semigroup 


{u’v*  I  i,j  >  1,  not  both  i  =  j  -  0}, 


(2.14) 


on  two  letters  u  and  v.  In  this  semigroup,  multiplying  two  elements  u‘‘v^'  and  u*’v^’  yields 
the  element  u*‘  v^'  .  I 


2.3  Basis  rules 

So  far,  we  have  ignored  the  basis  rules  of  a  program  in  order  to  concentrate  on  the  multi¬ 
plicative  structure  of  its  recursive  rules.  Unfortxmately,  we  pay  a  price  for  this  simplification 
when  we  come  to  study  the  query  containment  properties  of  a  program  that  has  its  recur¬ 
sive  rules  instantiated  by  nonrecursive  basis  rules.  It  is  not  always  enough  to  simply  study 
the  query  containments  on  a  program’s  set  of  recursive  top-down  expansions  *7(5).  The 
difficulty  arises  because  the  application  of  basis  rules  of  a  program  may  introduce  new  query 
containments  that  are  not  apparent  from  an  inspection  of  its  recursive  rules  only. 


Example  2.9  Consider  the  program  11  defined  by 

6  :  p(X)  :  -  «(X1) 

tt  :  p{X)  :  -  p(l)  t  •(XA). 


(2.15) 

(2.16) 


First,  we  focus  only  on  the  rule  expansion  semigroup  J{S)  generated  by  the  recursive  nde 
set  5  =:  {u}.  We  find  that  just  as  in  Example  2.7  above,  J{S)  is  isomorphic  to  the  free 
semigroup  U'  =  {u,uu,uuu,...}  generated  by  the  formal  symbol  u.  In  fact,  slightly  more 
is  true  in  the  present  example  and  in  Example  2.7.  Not  only  is  there  no  relation  u*  2; 
between  top-down  expansions  unless  i  =  j,  but  there  is  also  not  even  a  query  containment 
ti*  ^  uf  unless  i  =  j. 

However,  to  stop  here  would  leave  us  with  an  erroneous  picture  of  the  query  containments 
in  n.  The  actual  relation  p  that  n  defines  is  given  by  the  infinite  collection  of  conjunctive 
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queries 

6:p(X)  :-  «(XA). 

ti  :  p(X)  :  -  .(AA')  A  e(XA), 

«*  :  p(X)  :  -  «(A'A")  ft  e(AA')  ft  .(XA), 

together  with  the  general  query  for  every  t  >  3, 

u‘  :  p(X)  e(A(^~*)A(^))  ft  ft  •••  ft  e(ll')  ft  «(X1),  (2.17) 

Here,  we  have  used  the  notation  u*  to  stand  for  the  rule  u*  with  the  basis  rule  b  substituted 
for  its  p-$ubgoal.  l\e  see  that  in  11  itself,  we  have  the  quer^r  containments  u*  ^  b  for  every 
I  >  1:  the  required  containment  mapping  :  6  u*  is  given  by  defining  0,(p(X))  =  p(X) 
and  ^,-(e(XA)}  =  e(XA)  for  every  i  >  1.  The  consequences  for  efficient  query  processing 
of  the  program  n  are  drastic:  to  compute  the  relation  p,  we  may  apply  the  basis  rule 
once,  and  then  stop.  The  program  11  is  therefore  bounded  in  the  tense  of  Naughton  and 
Sagiv  (NaSa87].  I 

The  boundedness  of  the  program  11  in  Example  2.9  above  depends  upon  the  specific 
interaction  between  its  basis  and  recursive  rules.  In  particular,  if  we  substitute  a  different 
rule  for  the  basis  rule  b,  we  may  easUy  change  11  into  an  (inequivalent)  anboanded  program 
(i.e.  a  program  for  which  there  is  no  finite  depth  N  for  which  every  expansion  of  depth 
*  >  fir  is  contained  in  some  expansion  of  depth  <  fiT.)  For  example,  one  can  easily  check 
that  the  program  II'  =  {6',  a}  defined  by 

b':p(X)  1(11) 

«  :  P(*)  :  -  P(*)  »  •(“)• 

is  tmbounded. 

We  shall  have  occasion  to  contrast  a  recursive  rule  set  like  5  ss  {u}  (equation  2.16 
above),  which  may  or  may  not  yield  bounded  programs  depending  on  the  particular  basis 
rules  chosen  for  it,  with  recursive  rule  sets  S  that  yield  bounded  programs  ngardlest  of 
whatever  basis  rules  we  may  select  for  them.  We  call  a  rule  set  of  the  latter  type  uniformly 
bounded,  again  following  Naughton  and  Sagiv  (NaSaSSj. 
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Example  2.10  The  rule  set  S  =  {”>  containing  the  single  rule 

r  :  p(XY)  :  -  p(XA)  t  e(BT), 


is  uniformly  bounded.  I 


The  prograun  boundedness  and  uniform  boundedness  decision  problems  are  known  to  be 
imdecidable  even  for  quite  simple  classes  of  programs  [Abit89],  [GMSV'87],  [Vard88].  Even 
so,  for  certain  restricted  classes  of  programs,  useful  algorithms  have  been  developed  for 
studying  boundedness,  and  these  algorithms  can  be  fruitfully  applied  to  optimize  programs 
(Ioan85],  [Ioan89],  {NaSa87),  [NaSa88],  [RSU\^89],  (Sar89a].  In  the  following  chapters,  we 
shall  take  as  our  starting  point  the  ideas  of  boundedness,  the  rule  expansion  semigroup, 
and  rule  commutativity  {Ioan89],  [RSU\'89),  and  we  shall  hope  to  explain  how  classical 
ideas  from  mathematical  semigroup  theory  can  be  used  not  only  to  codify  and  simplify 
some  previous  results  in  the  query  optimization  literature,  but  also  to  give  us  new  results 
about  query  optimization  and  point  directions  to  new  issues. 


Chapter  3 


Free  rules 


In  the  present  chapter,  we  shall  again  restrict  ourselves  to  the  study  of  fizute  nonempty  rvlt 
sets  S  whose  members  are  linear  rccursirc  Datalog  rules  over  a  single  recursive  predicate  p. 
A  paradigmatic  singleton  rule  set  S  ^  {r}  would  consist  of  the  transitive  closure  rule 

r  :  p(XiX2)  :  -  P(XiA)  »  •(AXa).  (3.1) 

Again,  in  such  a  rule,  the  variables  appearing  in  the  heads  of  the  rules  are  called  distinguished 
variables^  while  those  appearing  only  in  rule  bodies  are  called  nondistinguished  variables. 
It  will  again  be  our  assumption  throughout  that  if  the  arity  of  the  recursive  predicate  p 
is  I  >  1,  then  the  head  predicate  of  every  rule  in  S  has  t  distinct  distinguished  variables 
occurring  in  it,  so  we  assume  that  the  head  predicate  of  every  rule  in  5  is  p(XiX2  *  *  *  Xt). 

In  a  more  complex  example,  we  might  have  a  rule  set  S  =  tltat  consists  of 

three  recursive  rules 

rj  :  p{XiX2X3)  :  -  p(X2AX3)  *  a(X,A)  (3.2) 

r,  :  p(XiX2ls)  :  -  p(CX3D)  t  t(BD)  t  •(AC)  t  i(X2B)  t  »(XiA)  (3.3) 

rs  :  p(Xjl2X3)  :  -  p(X3CB)  t  •(AC)  t  t(X2B)  t  •(XjA).  (3.4) 

Here,  amongst  the  three  rules  of  5  we  find  the  distinguished  variables  Xj,  Xa,  and  X3,  while 

the  nondistinguished  variables  are  A,  B,  C,  and  D. 

Suppose5  =  {ri,...,rik}  is  anile  set.  It  is  useful  to  pause  here  to  collect  the  preliininary 
information  contained  in  the  previous  chapter  into  a  single  paragraph.  We  have  seen  in  the 
previous  chapter  that  corresponding  to  every  rule  set  S  there  is  a  natural  rule  expansion 
semigroup  G  s  J{S)  whose  ‘‘multiplication”  is  defined  by  recursive  expansion  of  the  rules 
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in  S.  If  we  associate  with  every  finite-depth  top-down  recursive  expansion  of  the  rules  in 
5  a  corresponding  word  u;  over  the  formal  s>Tnbols  {ri, . . . , r*},  then  when  two  words  u’l 
and  u>2  simultaneously  stand  in  the  relationships  tt’i  ^  wj  and  U’j  i  tt;i,  we  call  them 
equivalent,  and  we  write  wi  s  The  relation  2:  defines  an  equivalence  relation  on  the 
set  of  all  finite  words  over  the  alphabet  {rj, . .  .,r*}  which  “respects  recursive  expansion:” 
for  if  tt’i  >  tt>j  and  tcj  v  then  we  have  verified  (in  the  course  of  proving  the  splicing 
Lemma  2.4)  that  wiui^  ^  wsw^.  The  set  of  all  top-down  expansions  of  a  rule  set  S  that 
are  in  the  same  equis-alence  class  as  a  particular  expansion  w  can  therefore  be  identified 
with  the  one  particular  member  w  of  that  class,  and  ( J(5),  >)  becomes  a  partiaUy  ordered 
semigroup.  Without  any  loss  of  generality  we  can  refer  to  the  “element  w  in  the  rule 
semigroup  J{S)”  as  a  shorthand  for  “the  equivalence  class  to  which  the  element  u*  belongs 
in  the  rule  semigroup  J{S).” 

3.1  Rule  factorization 

Opposite  to  the  recursive  expansion  of  rules  we  may  place  the  idea  of  rule  factorization. 
As  an  illustration,  we  return  to  our  rule  set  S  =  {ri,rj,r3},  above  (equations  3.2,  3.3,  and 
3.4).  In  fact,  we  can  simplify  the  structure  of  these  three  rules  considerably  by  introducing 
two  new  rules  a  and  6  defined  by 

«  :  P(t:t2X3)  :  -  p(X2AX3)  t  s(XiA) 
h  :  p(XjX2X3)  :  -  p(X2X3A)  k  t(XiA), 
for  which  it  is  easily  verified  that 

rj  2:  o 
fa  2!  <?h 
Tz  s:  aba. 

We  may  say  somewhat  loosely  that  the  rules  rj,  ra,  and  rz  admit  a  “factorization” 
into  the  simpler  rules  a  and  6.  Once  we  have  such  a  factorization  in  hand,  higher-order 
dependencies  between  the  rules  may  be  won  from  the  associative  law:  we  have  for  example 

rari  2:  (o*6)(tt)  2:  {a*){aba)  2:  r Jrs. 


/ 


/ 
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Summarizing,  we  can  say  that  the  rule  set  S  satisfies  the  algebraic  relationship  rjri  i  rjrs, 
and  that  this  algebraic  relationship  is  implied  by  the  given  factorization. 

More  formally,  let  5  =  {rj, . .  .ffc}  be  a  rule  set,  and  suppose  that  u'j  and  tt>j  are  two 
finite  words  over  the  formal  alphabet  S.  Then  we  shall  say  that 

tt>l  S  Wi 

is  an  algebraic  relationship  of  S  precisely  when  the  two  conjunctive  queries  obtained  by 
recursively  expanding  the  rules  of  5  according  to  the  words  tnj  and  u-a  are  equivalent.  A 
trivial  sdgebraic  relationship  is  one  in  which  the  two  words  ti»i  and  wj  are  identical.  If  r 
is  the  rule  obtained  by  syntacticaDy  expanding  nile  rj  by  the  rule  ra,  then  we  shall  say 
that  r  admits  the  factorization  rjra.  Of  course,  it  also  foUows  that  r  i  rira  if  r  admits 
the  factorization  rit,.  If  rj  and  ra  each  have  at  least  one  nonrecursive  subgoal  apiece, 
we  shall  call  the  factorization  nontrivial-,  otherwise,  the  factorization  is  sud  to  be  tnrioi. 
The  distinction  between  trivial  and  nontrivial  factorizations  will  not  be  important  to  us 
untU  Chapter  5.  If  5  and  T  are  rule  sets,  we  shall  also  say  occasionaUy  that  5  admUs  a 
factorization  over  T  if  every  element  r  in  5  is  either  an  element  of  T  or  alternatively  r 
admits  a  factorization  rir2  where  ri  and  r2  are  either  elements  of  T  or  recursive  expansions 
of  elements  of  T. 

3.2  Commutativity,  free  rules,  and  boundednesc 

Several  authors  [Ioan89],  [RSUV89]  have  pointed  out  that  the  complexity  of  recursive  query 
processing  can  be  reduced  when  elements  of  a  rule  set  satisfy  a  commutative  law  Tir2  :s  rr’’!- 
In  the  present  context,  we  prefer  to  think  of  a  commutative  law  as  nothing  more  than 
the  simplest  of  algebraic  relationships  that  a  given  rule  set  S  may  satisfy.  However,  every 
algebraic  relationship  satisfied  by  a  rule  set  will  lead  to  some  sort  of  redundancy  in  evaluation 
of  its  rules  which  a  query  optimizer  may  be  able  to  exploit.  The  questions  arise  therefore: 
not  only  “do  we  have  commutativity?"  but  more  generally  “what  are  aU  the  algebraic 
relationships  satisfied  by  this  rule  set?"  In  the  present  section,  we  hope  to  illustrate  a 
general  theory  that  can  be  applied  under  certain  circumstances  in  sudi  a  setting.  We  need 
to  make  two  de&utiozis: 

Definition  5.1  Let  S  be  a  rule  set  We  say  S  is  weakly  free  if  these  rules  satisfy  no 
nontrivial  algebraic  relationships. 
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Put  another  way,  a  rule  set  is  weakly  free  if  it  has  no  two  distinct  top-down  expansions  that 
mutually  contain  one  another. 

We  shall  also  be  interested  in  rule  sets  that  are  not  only  weakly  free,  but  also  satisfy  the 
stronger  condition  that  no  conjunctive  query  in  its  set  of  top-down  expansions  is  contained 
in  another  such  query: 

Definition  3.2  Lei  S  be  a  rule  set.  We  say  S  is  strongly  free  (or  more  simply,  bee)  if  the 
query  containment  relation  >  on  the  set  of  top^dovm  expansions  of  the  rules  of  S  is  trivial; 
i.e.,  no  such  query  is  contained  tn  any  other  (except  itself). 

It  is  useful  at  this  point  to  give  a  brief  catalog  of  examples. 

Example  3.3  The  simple  transitive  closure  rule 

«  :  p(XiX2)  :  -  p(XiA)  t  •(AXa) 

is  strongly  free.  To  see  this,  we  return  to  the  general  sth  recursive  expansion  of  the  rule  u 
(compare  with  equation  2.2,  above): 

:  p(XiX2)  :  -  p(Xii<‘~*))  t  »  •  •  •  *  e(A'A)  A  •(AX2). 

(We  identify  A**  with  A  so  that  =  u  in  our  notation). 

In  order  to  prove  that  the  rule  set  S  =  {u)  is  strongly  free,  we  must  show  that  there 
can  be  no  containment  mapping  0  :  ti*  u^'  unless  t  =  j.  So  suppose  that  ^  :  u*  u^  is 
a  containment  mapping.  Then  by  condition  2  on  we  have  ^(p(XiA(‘“^l))  =  p(XiAW“*J), 
and  ^(A^^“'))  =  Condition  3,  on  the  other  hand,  forces  ^•(AX2))  =  •(XX2),  so  that 

^(X)  =  h.  To  show  that  s'  must  be  equal  to  j,  it  is  simplest  to  rule  out  the  possibilities  that 
i>  j  and  t*  <  j  individually. 

First  suppose  that  j  —  s  =  j  >  0.  Then  by  using  condition  4  on  ^  repeatedly,  we  find  first 
that  s=  implies  ^a(*~*1)  =  and  so  on,  until  finally  we  conclude  that 

^A®)  =  But  since  we  already  know  that  ^A®)  =  A®,  we  conclude  that  y  - 1‘  =  «  s  0, 

a  contradiction. 

Alternatively,  suppose  that  y  -  s’  s  s  <  0.  Then  again  by  using  condition  4  on  ^ 
repeatedly,  we  find  first  that  0(A®)  =  A®  implies  ^(A^)  =  A^,  and  so  on,  until  finally  we 
conclude  that  ^(A^*'"*))  =  But  since  we  already  know  that  ^(A^^"*))  =  we 

conclude  that  i  —  1  =  y  -  1;  i.e.  s'  =  y,  again  a  contradiction. 
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A  nde  set  with  more  than  one  linear  recursive  rule  may  be  strongly  free. 

Example  3.4  The  rule  set  S  =  {c.d}  consisting  of  the  two  rules 

c  :  p(XiX2)  :  -  P(^i*)  * 

d  :  p(XtX2)  :  -  P(Xt*)  *  ®(A^2) 

is  strongly  free.  (See  definition  3.9  and  Theorem  3.10,  below,  for  more  discussion).  I 

Of  course,  a  rule  set  with  more  than  one  linear  recursive  rule  need  not  be  strongly  free: 
Example  3.5  The  rule  set  consisting  of  the  two  rules 

«  ;  p(XiX2)  :  -  p(Xii)  t  e(AX2) 

t;  :  p(XiX2)  :  -  p(AX2)  A  eCXiA) 

is  neither  strongly  nor  weakly  free.  (We  have  the  commutative  law  uv  n  uu— see  Exam¬ 
ple  2.8).  I 

We  have  said  that  a  rule  set  5  =  {ri, . .  ..r*}  is  uniformly  bounded  if  there  is  a  constant 
N  such  that  every  recursive  expansion  w  of  its  rules  is  contained  in  some  lecursive  expansion 
of  depth  <N.l£u  rule  set  is  not  uniformly  bounded,  we  shall  simply  call  it  unbounded.  From 
the  query  optimization  point  of  view,  boundedness  is  important  because  it  captures  the  idea 
of  when  a  recursive  rule  or  rule  set  may  be  replaced  by  an  equivalent  set  of  nonrecursive 
rules.  We  shall  see  below  how  the  idea  of  uniform  boundedness  arises  quite  naturally  in  the 
free  rule  setting.  First,  the  following  theorem  is  immediate. 

Theorem  3.6  Stron^y  free  rule  sets  are  unbounded. 

Proof  If  5  is  a  strongly  free  rule  set,  then  it  has  no  nontrivial  containments  between 
its  recursive  expansions.  In  particular,  there  is  certainly  no  finite  depth  N  for  which  every 
ijxpansion  is  contained  in  some  expansion  of  depth  <  JV^,  so  S  is  unbounded.  I 

More  importantly,  when  a  rule  set  factors  into  weakly  free  rules,  then  we  can  reduce  the 
study  of  all  its  algebraic  relationships  to  the  study  of  its  rule  expansion  semigroup: 
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Theorem  3.T  Let  S  be  a  rule  set  admitting  a  factorization  over  a  weakly  free  rule  set 
Then  S  satisfies  only  those  algebraic  relationships  that  are  implied  in  the  rule  expansion 
semigroup  J{S)  associated  with  this  factorization. 

Proof  Suppose  S  =  {n.-.-.r*}  admits  a  factorization  over  a  weakly  free  rule  set 

Then  every  rule  5  is  equivalent  to  some  recursive  expansion  of  rules  chosen  from 
^  =  {/i,  •  •  -  .  /f}-  To  capture  this  information  symbolically,  we  may  wTite  i  Wi  for  some 
formal  word  Wi  over  the  symbols  Now  suppose  r  s  j  is  some  algebraic  relationship 
satisfied  by  the  rules  of  S  (here  r  and  s  are  words  over  the  alphabet  S).  Replacing  every 
occurrence  of  each  rj  in  r  and  s  by  its  corresponding  word  iCi  over  we  find  that  the  the 
resulting  two  words  over  ^  must  be  identical  (for  otherwise  ^  would  not  be  weakly  free). 
Therefore  r  s  a  is  a  consequence  of  the  given  factorization,  so  our  proof  is  complete.  I 

Even  more  can  be  said  when  we  have  a  strongly  free  factorization. 

Theorem  3.8  Let  S  be  a  rule  set  admitting  a  factorization  over  a  strongly  free  rule  set 
T.  Then  S  satisfies  only  those  algebraic  relationships  implied  by  this  factorization,  and  the 
query  containment  relation  t  on  the  set  of  top-doum  expansions  of  the  rules  of  S  U  an 
equivalence  relation. 

Proof  Because  every  strongly  free  rule  set  is  also  trivially  weakly  free,  all  that  needs 
to  be  checked  is  that  if  si  and  S2  are  two  expansions  of  the  rules  in  5  and  si  >  sa,  then 
*2  ^  *1  t*  *1*®  true.  Our  argument  proceeds  exactly  as  in  Theorem  3.7,  above: 

Suppose  S  =  {rj,. .  ..r*}  admits  a  factorization  over  a  strongly  free  rule  set  Then 
every  rule  rj  €  5  »  equivalent  to  some  recursive  expansion  of  rules  chosen  from  = 
{/i.  •  •  • .  ft}>  fonnal  word  over  the  symbols  for 

each  i.  Now,  if  Zj  t  zj  is  some  query  containment  between  expansions  of  the  rules  in  5, 
then  we  replace  each  rt  in  Zj  and  Z}  by  its  corresponding  word  va  over  Again,  we  find 
the  the  resulting  two  words  over  ^  must  be  identical  (for  otherwise  ^  would  not  be 
strongly  free).  So  in  fact  zx  s  zj,  and  in  particular,  zj  ^  zi,  as  we  were  required  to  show. 

I 

That  is  to  say,  if  a  rule  set  S  —  {ri, . .  .,r*}  can  be  factored  into  strongly  free  rules  and 
we  have  a  conjunctive  query  containment  wx  >  tej  between  two  top-down  expansions  of  the 
rules  of  S,  then  in  fact  we  also  have  tu:  ►  lox.  Thus  the  study  of  all  query  contunments 
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satisfied  by  the  rule  expansions  of  S  is  reduced  to  the  study  of  the  associated  subsemigroup 
of  the  free  semigroup  on  the  sj-mbols  {fj , . . . ,  r*}.  We  shall  return  to  this  idea  below. 

We  may  also  think  of  our  theorems  as  casting  light  on  the  following  general  problems; 

1.  Given  a  rule  set  5,  fully  characterize  the  query  containment  partial  ordering  >  on  its 
set  of  top-down  expansions.  WTiat  is  the  computational  complexity  of  determining 
whether  given  top-down  expansions  stand  in  the  relation  v  trj? 

2.  Identify  classes  of  rule  sets  for  svhich  the  first  question  can  be  answered  efficiently. 

The  first  problem  is  nard:  it  is  NP-complete  to  determine  whether  tt»i  v  u-j  for  general 
queries  [ChMe77].  Yet  Theorem  3.8  points  us  in  the  direction  of  classes  of  rule  sets  for 
which  query  containment  problems  admit  efficient  solution  by  semigroup  techniques.  For 
example,  consider  the  following  class  of  rule  sets,  of  which  the  rule  set  in  Example  3.4, 
above,  is  a  representative.  We  need  a  preliminary  definition. 

Definition  3.9  Let  k  >  0  be  an  integer.  A  left-right  transitive  closure  rule  is  a  linear, 
singte~edb  rule 

pCA'iXj):  &  ek  &  '  •  -  &  ei  &  eo 

where  either  ei  =  e(.4j4^^))  or  ei  =  e(j4(*5.4),  and  in  general  either  e^  =  e(j4f*“*)i4(*))  or 
ei  =  for  2  <  i  <  k;  finally  either  eo  =  e(A}X)  or  eo  =  e(AAj). 

***^ple»  rules  c  and  d  of  Example  3.4  are  both  left-right  transitive  closure  rules. 

Theorem  3.10  Let  S  be  a  rule  set  consisting  of  a  finite  collection  of  left-right  transitive 
closure  rules.  Then  two  queries  w,  and  wj  in  the  set  of  top-down  expansions  of  the  rules 
of  S  can  be  tested  for  toi  ^  toj  tn  linear  time;  moreover,  whether  there  are  any  two  distinct 
quertes  wi  and  to?  sudi  that  wi  >;  toj  can  be  determined  in  polynomial  time. 

Proof  Suppose  r  6  5  is  a  left-right  transitive  closure  rule  with  t  >  0  nonrecursive 
subgoals  Ci, . . ., Ck  and  a  (4  -I-  l)st  subgoal  eo  as  in  the  previous  definition.  Then  we  Maim 
r  is  a  recursive  expansion  of  the  rules  c  and  d  of  Example  3.4,  above:  in  fact,  we  claim  that 
r  2:  fik-t-iffk  -  -  -  A,  where 


^  _  e  if  ej  =  e(j4(‘”*).4(*)) 
if  ei  =  e(i4^*).4{*“*)) 
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for  1  <  t  <  ib,  and 

^  _  I  c  if  eo  =  e(A'j-4) 

^  \  d  if  eo  =  e(.4A’2). 

The  claim  may  be  proved  by  induction  on  the  integer  k.  First  note  that  if  fc  =  0,  then  r  is 
either  the  rule 


p(XiX2)  :  “  P(XiA)  *  •(X2A), 


in  which  case  t*  i  c,  or  alternatively  r  is  the  rule 


p(XlX2)  ;  -  P(XlA)  k  •(AXj), 


in  which  case  r  s  d.  Inductively,  we  now  assume  that  the  claim  is  valid  for  all  left-right 
transitive  closure  rules  t  with  ifc* values  less  than  some  fixed  positive  integer  K ,  and  consider 
the  claim  for  the  value  K.  Let  r'  be  the  rule  obtained  from  r  by  deleting  its  rightmost 
EDB  subgoal  eo  and  decreasing  all  the  superscripts  on  its  nondistinguished  .4-variables  by 
one  (the  variable  A  itself  is  to  be  replaced  by  the  distinguished  variable  A'j).  Then  r'  is 
itself  a  left-right  transitive  closure  rule  with  one  fewer  EDB  subgoal  than  r.  The  inductive 
hypothesis  therefore  applies,  and  r'  is  a  recursive  expansion  of  the  rules  e  and  d  as  in  the 
rlaiTT.  Statement  above.  Now  if  the  occurrence  of  eo  in  r  is  0(4X2),  then  by  the  definition 
of  syntactic  expansion,  the  rule  r  is  obtained  by  recursively  expanding  the  rule  r'  by  the 
rule  c,  and  if  alternatively  the  occurrence  of  eo  in  r  is  e(X2A),  then  the  rule  r  is  obtained  by 
recursively  expanding  the  rule  r*  by  the  rule  d.  We  therefore  have  verified  the  clum  above. 

Now,  because  the  rule  set  T  =  {e,d}  is  strongly  free  (Example  3.4),  Theorem  3.8 
guarantees  that  a  conjunctive  query  containment  wj  >:  wj  can  be  tested  by  writing  down 
their  respective  factorixations  over  T,  and  checking  whether  these  two  words  are  equal. 
Because  two  words  wi  and  wt  over  the  formal  alphabet  {c,  d}  can  be  tested  for  equality  in 
tiTn*  litwnr  in  Itoil  +  |i»2li  *  given  containment  »i  >;  ti>2  «1*®  can  be  tested  in  linear  time. 

To  test  whether  there  exist  queries  wi  and  toj  such  that  wi  >  wt.  Theorem  3.8  again 
allows  us  to  reduce  the  problem  to  one  about  words  over  the  formal  alphabet  T.  More 
precisely,  we  face  the  following  problem:  given  a  finite  set  W  of  words  over  the  alphabet 
T  =  {c,d},  how  can  we  recognize  whether  there  is  a  single  word  w  6  W'  that  adnuts  two 
distinct  factorizations  into  the  words  W?  In  the  present  context,  the  words  W  arise  as 
factorizations  of  the  rules  in  S  into  the  rules  c  and  d  of  T,  and  we  see  that  there  will  be  a 
nontrivial  query  containment  t»j  >  precisely  when  one  recursive  expansion  of  the  rules 
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S  “decomposes”  in  two  different  ways.  'WTien  there  is  no  such  word  w,  the  set  W  is  called 
a  code.  Because  there  are  known  polynomial  time  algorithms  for  recognizing  codes  in  free 
semigroups  (see  |Speh75].  and  the  following  section,  for  more  details),  the  present  proof  is 
complete. 

I 

Theorem  3.10  is  meant  primarily  as  an  illustration  of  a  general  technique— in  fact,  there 
are  many  theorems  like  Theorem  3.10:  our  ability  to  state  them  depends  only  on  our  ability 
to  find  sets  of  strongly  free  rules  and  describe  the  rule  sets  they  “generate.” 

3.3  Codes  and  free  rule  set  recognition 

To  explore  these  ideas  a  bit  further  and  pick  up  some  semigroup  theory  along  the  way,  it  is 
useful  to  consider  another  example.  Consider  the  rule  set  S  =  {ri,r2.f3}  given  by 


r,  :  p(XiX2) 

:  -  p(XjA)  t  e(X2A) 

(3.5) 

ra  :  p(XiX2) 

:  -  p(XiC)  k  e(CB)  t  e(lB)  k  e(AX2) 

(3.6) 

ra  :  p(XiX2) 

:  -  p(XiB)  k  e(BA)  k  e(X2A) 

(3.7) 

These  three  rules  are  left-right  transitive  closure  rules.  As  such,  they 

can  be  expressed 

as  recursive  expansions  of  the  strongly  free  rules  c  and  d  of  Example  3.4,  above:  we  have 

ri  2r  c 
r}  2:  ded 
fa  2:  ed. 

Theorems  3.8  and  3.10  allow  ns  to  make  the  assertion:  any  conjunctive  query  containment 
t»i  i  tfa  satisfied  in  the  set  of  all  top-down  expansions  of  the  rules  in  5  is  in  fact  an  algebraic 
relationship  ii?i  2:  t»a;  moreover,  this  algebraic  relationship  is  necessarily  a  consequence  of 
the  given  factorization.  Thus  the  question:  “do  the  rules  {^1,  rj,  ra},  when  expanded,  satisfy 
any  nontrivial  conjunctive  query  containments  >  tuj?”  has  been' effectively  reduced  to 
the  purely  semigroup-theoretical  question:  “is  the  subsemigroup  of  the  free  semigroup  on 
the  formal  alphabet  {c,  d}  generated  by  the  set  of  word,  {c,  ded,  ed)  itself  a  free  semigroup?” 
The  power  of  the  techniques  described  above  depends  upon  the  fact  that  semigroup  theorists 
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have  OTdied  such  questions  in  souse  detail,  and  ha,e  jiven  algorithms  that  may  be  fruitfuliv 
carried  over  to  the  query  optimization  setting. 

Before  proceeding,  we  answer  the  question  asked  above.  Because 

Tara  2:  (cd){cd)  2r  c{dcd)  2  r,rj, 

the  subsemigroup  of  the  free  semigroup  on  the  letters  c  and  d  which  is  generated  by  c,  dcd, 
and  ed  is  not  itself  free.  Equivalently,  we  can  say  that  J[S)  is  not  a  free  rule  expaiision 
semigroup,  because  it  satisfies  the  algebraic  relationship  r|  2  r,rj. 

The  preceding  discussion  suggests  a  general  question:  how  hard  is  it  to  recognize  whether 
a  given  rule  set,  once  factored  into  strongly  free  rules,  admits  any  algebraic  relationships?  As 
we  remarked  in  Theorem  3.10,  above,  the  problem  is  exactly  the  so-called  “code  recognition 
problem”  that  has  been  completely  solved  in  the  free  semigroup  setting  by  Spehner  [SpehTo], 
who  has  not  only  given  a  beautiful  polynomial  time  algorithm  for  the  decision  problem,  but 
has  also  given  a  method  that  can  be  adapted  [LallTO]  to  construct  an  abstract  presentation 
of  the  associated  rule  expansion  semigroup.  Other  algorithms  for  testing  whether  a  finite  set 
of  finite  words  over  some  alphabet  A  form  a  code  have  been  given  ([Mark62],  [Blu63b’),  but 
we  shall  prefer  Spehner’s  algorithm  because  it  is  the  most  easily  explained  and  analyzed. 
Note  that  as  a  special  case,  we  may  decide  commutativity  in  polynomial  time  for  rule  sets 

5  that  factor  into  strongly  free  rules.  Recognizing  rule  commutativity  for  general  rules 
appears  to  be  difficult  [Sar89b]. 

Here,  we  shall  only  present  Spehner’s  algorithm  in  enough  detaU  so  that  we  can  verify 

it  can  be  carried  out  in  polynomial  time.  In  particular,  we  want  to  prove  the  following 
generalization  of  Theorem  3.10: 

Theorem  3.11  Suppose  S  is  a  rule  set  whose  members  have  been  factored  over  a  strongly 
free  rule  set  Then  the  following  may  be  decided  in  polynomial  time:  (i)  recognizing  com- 

mutaUvity;  and  (it)  more  generally,  recognizing  whether  there  exists  any  nontrivial  algebraic 
relationship  holding  amongst  the  rules  in  S. 

Proof 

Suppose  S  —  {ri,. . ,,r*}  admits  a  factorization  over  such  a  strongly  free  rule  set 
Then  every  rule  r,  €  5  is  equivalent  to  some  recursive  expansion  of  rules  chosen  from  a 

stronglj  free  set  of  rules  :F  -  {/i, . and  we  may  write  each  as  some  formal  word 
Wi  over  the  symbols 
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To  recognize  whether  two  given  rules  r,  and  rj  in  S  commute,  we  can  simply  write  down 
their  factorizations  over  in  the  two  orders  r,rj  and  and  check  to  see  if  the  resulting 
words  are  identical:  by  Theorem  3.8,  nothing  more  needs  to  be  said. 

To  recognize  whether  there  exists  some  nontrivial  algebraic  relationship  amongst  the 
rules  in  5,  we  murt  decide  w’hether  the  words  form  a  code  in  the  free  monoid  over 
the  alphabet  (Recall  that  the  free  monoid  over  ^  is  just  the  free  semigroup  over  T 
with  a  special  identity  element  adjoined  to  it.  Multiplication  here  corresponds  to  string 
concatenation,  and  we  can  think  of  the  identity  as  representing  the  empty  word  over  ^.) 

We  now  sketch  definitions  preliminary  to  Spehner’s  solution  to  the  code  recognition 
problem  as  it  applies  to  the  problem  at  hand.  We  follow  Lallement  [Lall79].  Let  W  be  the 
submonoid  of  generated  by  the  words  u’j.  We  call  a  pair  (u,  r)  €  x  a  W-pair  if 
there  exists  a  w  €  W  such  that  Wi  =  uu»r  for  some  u’i,  with  uw  ^  1  and  wv  ^  1.  We  denote 
the  W-pair  (u,v)  by  the  symbol  u  — ►  v.  A  pair  (t4,v)  £  k  T*  is  called  W-connected 
(notation  u  ^  v)  if  there  is  a  sequence  u  =  tioi  Ui, . . . ,  Un  =  v  such  that  u»  — ►  for  every 

0  £  *  <  Finally,  we  define  r(W),  the  graph  ofWy  as  the  graph  having  as  set  of  vertices 
the  set  U{W)  =  {u;  €  i  =>  1  and  1  =>  tr},  and  as  a  set  of  directed  edges  the  M'-pairs 
(ti,r)  with  u,  t;  €  U{W). 

For  example,  we  return  to  the  factorization  {c  ded  =  ti;2,  cd  =  of  the  rules 

3.5,  3.6,  and  3.7,  above.  To  calctilate  the  associated  W-pairs,  it  is  simplest  if  we  consider 
each  word  Wi  individually,  and  build  the  small  table  in  Table  3.1. 


Idcdl 
d^cd- I 


Table  3.1:  The  W*pairs  derived  from  the  set  of  words  {c,^cd,c 
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Apparently  r(W*)  =  The  graph  T{\y)  is 


The  result  we  need  from  Spehncr  can  now  be  succinctly  stated: 

W  is  free  over  the  words  u/j  iff.  the  graph  of  W  is  trivial  (i.e.  if  is  identically  1*^1). 

In  otir  tunning  example,  the  graph  T{W)  is  not  trivial,  so  we  have  here  an  alternative 
proof  that  the  submonoid  generated  by  c,  ded,  and  cd  is  not  a  code. 

Finally,  we  turn  to  the  time  analysis  of  using  Spehner’s  method  to  decide  part  (ii)  of 
the  theorem  statement.  First,  note  that  because  the  monoid  W  is  generated  by  a  finite  set 
of  words  it  is  in  fact  a  regular  language  for  which  we  may  decide  the  membership  of  any 
given  word  xv  in  time  polynomial  in  iu;|  -r  Itt’j!.  Next,  note  that  the  equation  iCi  =  titrv 
entails  |ti;|  <  iwij,  so  we  may  therefore  determine  whether  a  given  possible  candidate  for  the 
role  of  w  in  a  decomposition  Wi  =  uu;i»  satisfies  w  £  W  in  time  polynomial  in 
generate  all  the  W"*pairs  is  then  also  a  polynomial-time  problem,  because  we  simply  can  try 
for  each  word  Wi  all  the  0(|u;»p)  possible  decompositions  Wi  =  uwv,  checking  in  each  case 
whether  w  £W.  The  resulting  graph  r(W^)  can  then  be  scanned  for  connected  components 
using  simple  techniques,  and  checked  for  a  tri%*iality  in  a  final  step. 


3.4  Presenting  query  containments 

If  a  rule  set  5  factors  over  a  strongly  free  rule  set,  then  Spehner’s  method  can  be  extended  to 
give  an  abstract  presentation  of  the  rule  expansion  semigroup  J{S).  Such  a  presentation  is 
in  effect  a  complete  mathematical  specification  of  all  the  redundancy  encotmtered  between 
the  recursive  expansions  of  the  rules  5,  in  the  sense  that  all  query  containments  between 
recursive  expansions  will  be  algebraic  consequences  of  the  given  presentation. 

To  describe  the  method  we  need  to  pick  up  some  more  elementary  results  and  definitions. 
Again,  we  follow  Lallement  [Lall79]. 
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Let  M  be  a  submonoid  of  the  free  monoid  A"  on  an  alphabet  A.  If  we  let  Af  ^  stand  for 
the  semigroup  A/  -  {!},  then  one  has  the  following  theorem  [Lall79i. 

Theorem  3.12  Every  submonotd  M  of  a  free  monoid  has  a  unique  minimal  set  of  gener^ 
ators  C  =  -  (Af^)^.  The  set  C  is  called  the  base  of  M. 

Example  3*13  In  the  free  monoid  -4“  on  the  alphabet  A  —  the  ^ubmonoid 

M  = 

has  as  its  base  the  set  C  =  {a^,a^}»  Note  that  M  is  not  free  over  C:  for  example  the  word 
a®  admits  the  two  factorizations  and  a® a®.  I 


Restating  the  definition  of  a  code  in  terms  of  bases  we  have 

Definition  3.14  A  subset  C  of  a  free  monoid  .4“  is  called  a  code  over  A  if  C  is  the  base 
of  a  f  ee  submonoid  M  of  A* . 

Rettiming  to  the  notation  and  hjT)othesis  of  Theorem  3.11,  suppose  that  every  member 
r,  of  a  rule  set  S  is  equivalent  to  some  recursive  expansion  of  rules  chosen  from  a  strongly 
free  set  of  niles  .F  =  {A, . . . ,  ft]*  Then  as  before,  we  may  write  each  as  some  formal  word 
Wi  over  the  symbols  Let  W  =  Ui^t»  let  A/  be  the  monoid  generated  by  the  words 
Wi  in  the  free  monoid  F*.  Also,  we  shall  suppose  for  the  moment  that  redundant  generators 
u*t  have  been  eliminated  so  that  the  words  Wi  are  in  fact  a  base  for  the  submonoid  W*  of 
F"*  We  shall  return  to  the  issue  of  eliminating  redundant  generators  below. 

If  we  construct  the  graph  T  =  r(W)  from  the  W-pairs  as  in  Theorem  3.11  and  then 
find  that  the  graph  T  is  trivial,  then  it  follows  that  the  rule  set  5  is  free  and  that  5  satisfies 
no  nontrivial  query  containments.  It  is  our  goal  to  explain  how  Spehner’s  method,  in  the 
case  where  the  graph  T  is  nontrivial,  can  be  extended  to  give  an  abstract  presentation  of 
the  semigroup  J[S).  Because  the  algorithm  in  question  is  essentially  due  to  Spehner  we 
shall  omit  the  proof  of  correctness  and  shall  satisfy  ourselves  with  a  formal  description  of 
the  technique  and  an  example. 

First,  we  need  some  more  definitions,  which  we  take  from  [Lall79].  If  (u,  v)  is  a  W-pair, 
then  by  definition  there  exist  z  €  W  and  y  €  such  that  uyv  =  z*  We  shall  say  that 
the  pair  (z,y)  is  produced  by  the  W-pair  (ti,  v).  If  uotUi, .  • .  ,Un  is  a  sequence  of  words  in 
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F-  such  that  each  (u.,u„i)  i,  a  H'.pair,  and  if  (j,.,,,)  j,  *  produced  bv  («.,u.  ,)  for 
I  =  0,1,..., n -  1.  then  the  relation  ‘ 

*2  •  • .  =  yoiiyj . . . 

is  called  a  relation  produced  by  the  W-$equence  uq,  «i,  • . . ,  u„. 

A  umpk  If-crctf  i,  ju,.  . 

such  that  the  pairs 

(1.  “l).  («1,  tir), .  •  • ,  (Un-l,  u„),  (tt„,  1) 

are  W-pairs  and  «.  #  1  for  f  =  We  write  a  simple  W-circuit  (1,«, . 1). 

At  the  heart  of  Spehner’s  method  is  the  foUowing  result  :Speh73]. 

Theorem  5.15  Let  H'*  be  a  submonoid  of  F'  with  base  W,  and  let  ,  i) 

6e  a  stmple  W.ciremt  in  the  graph  cfT.  Then  the  relation  produced  from  thrs  W-ci^Z  is 
a  relation  satisfied  in  Moreover,  every  relation  in  W*  is  an  algebraic  consequence  of 
relations  produced  by  simple  W-circuiU  of  the  graph  ofT. 

To  illustrate  the  theorem,  we  return  once  more  to  the  rules 
ri  :  p(XiX2)  :  -  p(XjX)  t  •(Xj*) 

:  PfXiXa)  :  -  p(XiC)  t  efCB)  k  *(18)  t  e(AX2) 

*■3  :  p(XiX2)  :  -  p(XiB)  k  e(BA)  k  •(X2i) 

from  Section  3.3.  These  rules  could  be  written  as 

rj  2:  e 
**2  2:  dcd 
^3  2:  ed, 

where  the  rules  c  and  d  are  taken  from  Example  3.4.  The  simple  IT-circuits  in  the  graph  T 
4re  ^  ^ 


{hd,l) 

{hd,d,l) 

{hd,d,d,l) 

(l,d,d,d,d,l) 


^  / 
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for  which  the  corresponding  produced  relations  are 

rafs  i  rjfj 
rarjfj  i  rirjra 
farifara  n:  rjrarirj 
rafirjrifjra  2:  rifjrjrjrirj 

Simplifying  these  expressions  according  to  whether  an  odd  or  even  number  of  d’s  are  chosen 
to  occur  in  the  U'*ciTcuit,  we  obtain  the  presentation  relations 

V3(rirj)‘‘^*  2:  (rirj)’-^*r3, 

where  t  is  allowed  in  the  range  t  >  0. 

All  query  containments  satisfied  by  the  rules  rj  and  ra  are  consequences  of  the  given 
relations  above. 


Chapter  4 


Singleton  rule  sets 


Given  the  computational  power  that  comes  along  with  recognizing  that  a  given  rule  set 
factors  into  free  rules,  we  may  ask:  how  hard  is  it  to  factorize,  and  how  hard  is  it  to  recognize 
free  nile  sets?  In  the  present  chapter,  we  shall  begin  a  general  study  of  these  questions  by 
focusing  on  rule  sets  S  that  consist  of  a  single  linear  recursive  rule  r.  Not  surprisingly,  we 
find  points  of  contact  between  the  ideas  of  freeness  and  program  boundedness  here.  First, 
it  is  not  difBcult  to  prove: 

Theorem  4.1  Let  S  =  {r}  be  a  singleton  nde  set.  Then  S  is  uniformly  bounded  if  and 
only  if  there  is  no  single  nontrivial  containment  r‘  v  H  with  i  <  j. 

Proof  A  slightly  weaker  form  of  the  theorem  was  proved  by  Naughton  [Naug86a],  but 
the  splicing  lemma  allows  us  to  give  a  simpler  proof. 

If  5  is  uniformly  bounded,  then  by  definition  it  has  nontrivial  containments  r*  ^  with 
t  <  j. 

For  the  converse,  first  note  that  because  r  2:  r,  we  may  apply  the  splicing  lemma 
repeatedly  to  prove  first  that  ^  and  then  inductively  that  r*'*’*  ^  for  every 
positive  integer  a. 

Next,  we  that  that  r*  >  for  every  positive  integer  m. 

We  may  prove  the  claim  by  induction  on  m.  First,  when  m  =  1,  we  have  s  r* 

so  the  claim  is  valid.  Now  suppose  the  claim  is  valid  for  all  values  m  strictly  less  than  some 
positive  integer  f,  and  consider  the  case  m  =  t.  Fkom  the  induction  hypothesis,  we  have 
r*  V  Adding  the  constant  a  =  ;  -  s*  to  the  superscripts  on  both  sides,  we 

obtain  r>  >  Since  by  assumption  r*  ►  H,  we  can  use  the  transitive  property  of 
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containment  (Lemma  2.3)  to  conclude  that  r‘  v  completing  the 

inductive  step  and  the  proof  of  the  claim. 

To  finish,  we  must  show  that  there  is  a  constant  N  such  that  for  all  ir  >  .V,  we  have 
r*  V  r*  for  some  I  <  N;  i.e.,*5  is  uniformly  bounded.  We  shall  see  that  the  choice  N  =  j 
works.  In  proof,  let  /  >  j  be  any  positive  integer,  and  use  the  division  algorithm  to  W7ite 
/  -*  «  =  g(j  -  i)  f  r  for  some  quotient  j  >  1  and  remainder  r  satisfying  0  <  r  <  (y  -  i). 
By  the  claim  above,  r*  ^  Adding  the  constant  a  =  r  to  the  subscripts  on  both 

sides,  we  obtain 

i.e.,  r*'*''  V  r*.  Note  that  t  +  r  <  t  +  j  -  i  =  y ,  as  required.  Since  I  was  completely  arbitrary, 
this  completes  the  proof.  I 

We  find  therefore  that  recognising  whether  a  singleton  rule  set  is  free  is  closely  related 
to  the  so-called  “uniform  boimdedness  problem  for  linear  single  rule  programs  (sirups)” 
[Kane88]  that  has  attracted  various  researchers  [Ioan85],  [JAN87],  [NaSa88],  [Vard88].  Re¬ 
cently,  Vardi  [Vard90j  has  announced  an  algorithm  for  the  linear  boundedness  decision  prob¬ 
lem  for  linear  sirups,  and  the  uniform  boundedness  problem  can  be  thought  of  as  a  special 
case  of  his  result.  However,  it  is  important  to  note  that  the  ideas  of  (non)&eeness  and 
uniform  boundedness  do  not  exactly  coincide,  because  it  is  possible  for  a  rule  to  be  simul¬ 
taneously  imbounded  and  not  strongly  free: 

Example  4.2  The  rule 

r  :  p(XiX2)  :  -  p(Xil)  k  •(iXa)  I  •(Al) 


is  unbounded  and  weakly  free,  but  not  strongly  free.  We  can  offer  only  a  partial  explanation 
using  the  tools  we  have  developed  so  far.  First,  it  is  possible  to  see  directly  that  r  is  not 
strongly  free,  because  the  derived  mapping  given  by  putting  =  AW  =  A 

suffices  to  define  a  containment  mapping  ^  -*  r.  Therefore  r  has  a  nontrivial  query 

containment,  and  is  not  strongly  free.  On  the  other  hand,  to  show  that  there  can  be  no 
containment  mapping  ^ :  r*  -»  H  with  t‘  <  y  (that  is,  to  show  r  is  not  uniformly  bounded, 
in  light  of  Theorem  4.1),  is  not  so  easy.  It  will  suffice  to  observe  that  r  has  an  unbounded 
head  chain,  and  that  r  has  no  cut,  but  here  we  anticipate  ourselves.  We  must  wait  for  oc' 
proof  of  Theorem  4.15,  bdow,  to  obt^  a  complete  explanation.  I 
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In  earlier  work,  [Vard88]  and  IGMSV87]  show  (among  other  things)  that  linear  monadic- 
recursive  and  binary-recursive  rules  can  be  tested  for  uniform  boundedness,  and  that  bound¬ 
edness  for  the  latter  problem  is  NP-complete.  Abiteboul  ,’Abit89]  has  shown  that  bound¬ 
edness  is  undecidable  for  programs  with  a  single  recursive  rule  with  nonlinear  rectirsion 
allowed,  but  his  techniques  do  not  apply  either  to  the  study  of  uniform  boundedness,  or  to 
the  case  of  linear  recursive  rules. 

Our  first  result  in  this  area  is  a  partial  one  about  “containment  depth:” 

Theorem  4.3  Let  r  be  a  linear  recursive  Datalog  rule.  Then  there  is  a  constant  K  depend- 
ing  only  on  r  such  that  if  the  ith  recursive  expansion  r*  contains  the  jth  recursive  expansion 
rt  for  some  j  >  i>  1,  then  r*  also  contains  r^'  for  some  j'  satisfying  i  <  j'  <  Ki. 

We  shall  prove  the  theorem  in  stages  through  the  several  intermediate  results  below. 

Perhaps  more  important  than  the  containment  depth  result  itself  is  the  notation  and 
preparatory  lemmas  that  we  shall  develop  in  proving  it.  Because  a  substantial  gap  ex¬ 
ists  between  what  boundedness  information  can  be  known  at  a  theoretical  level  and  what 
techniques  can  be  efficiently  implemented  in  a  real  system,  some  recent  research  atten¬ 
tion  has  been  given  to  polynomial-time  testable  sufficient  conditions  for  boundedness.  In 
the  final  three  sections  of  the  present  chapter,  we  shall  take  up  the  study  of  efficiently 
testable  sufficient  conditions  for  singleton  rule  set  to  be  unbounded,  and  we  shall  arrive 
at  a  polynomial-time-testable  sufficient  condition  for  unboundedness  that  simultaneously 
subsumes  several  earlier  criteria  for  this  problem. 


4.1  Notation  and  preliminaries 

We  shall  first  introduce  some  slightly  nonstandard  notation  for  linear  recursive  rules.  After 
TTi^lfing  our  formal  definitions,  we  shall  pause  to  relate  our  notation  to  our  earlier,  more 
familiar  representations  of  linear  recursive  rules. 

Let  n  be  an  arbitrary,  fixed  positive  integer.  Let  Pn  *  {1»2, ...n},  and  let  U  = 
{A, .}  be  an  arbitrary  finite  set  of  symbols  disjoint  from  Pn-  Let  V  =  Pn  U  U.  A 
linear  single  rule  program  (sirup)  of  arity  n  is  a  pair  Q~  Qo  —  [h,  S)  where  h  is  a  word  of 
length  n  over  the  alphabet  V,  and  5  =  {ei, . .  ..e*}  is  a  (possibly  empty)  finite  set  of  words 
also  over  the  alphabet  V.  When  a  linear  sirup  Q  has  at  least  one  word  ei  €  f ,  we  shall 
also  require  all  such  words  to  have  the  same  fixed  length  s  =  |eii.  When  speaking  of  2, 
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we  shall  call  h  its  recursive  subgoal,  the  Cj’s  its  EDB  subgoals,  s  its  EDB  subgoal  arity,  V 
its  variables,  I>„  its  distinguished  variables,  and  U  its  nondistinguished  variables.  We  shall 
find  It  convenient  to  write  a  linear  siiup  2  as  a  string  of  Ar  —  1  words  over  the  alphabet  V, 
where  we  understand  the  leftmost  word  to  represent  h,  and  where  the  remaining  words  (if 
any)  define  Cj  through  e*,  respectively. 

As  a  first  illustration  of  our  system  of  notation,  consider  again  the  familiar  transitive 
closure  rule  for  directed  graphs 

path(XiX2)  :  -  path(Xil)  t  edge(AX2). 

In  our  system  of  notation,  we  w'ould  write  this  rule  more  economically  as 

lA  A2, 

suppressing  the  head  predicate  entirely  and  all  predicate  names;  also,  we  have  removed 
the  dummy  distinguished  variable  symbol  X,  replacing  Xi  and  X2  by  the  sjrmbols  1  and 
2,  respectively.  It  is  implicit  in  our  representation  that  the  head  variables  of  our  rule  are 
distinct  symbols,  and  we  shall  always  make  this  assumption,  as  we  have  in  earlier  chapters. 
Because  the  word  lA  is  written  first,  we  tmderstand  it  to  represent  the  recursive  subgoal 
fc;  also  implicit  in  our  representation  are  the  facts  that  n  =  lh|  =  2,  that  i  =  1,  and  that 
ej  =  A2  is  the  only  EDB  subgoal  of  Q. 

For  a  second  example,  consider  the  rule 

t(XY2)  :  -  t(XWZ)  t  •(WY)  t  •(WZ)  k  €(ZZ)  t  «(ZY), 

which  in  [Naug86a]  is  given  as  an  example  of  a  uniformly  bounded  Datalog  rule  that  nev¬ 
ertheless  possesses  an  unbounded  head  chain  (see  Section  4.4).  We  would  write  Naughton's 
rule  as 

li43  A2  AZ  33  32, 

where  we  have  changed  the  nondistinguished  variable  W  into  an  .4  and  have  substituted 
the  symbols  1,  2,  and  3  for  the  distinguished  variables  X,  Y,  and  Z,  respectively. 

In  order  to  refer  to  particular  letters  in  a  word  of  a  linear  sirup,  we  use  a  selection 
operator  Oi  to  pick  out  the  sth  letter  of  a  word.  For  example,  in  our  representation  of 
Naughton’s  linear  sirup  above,  we  have  03(h)  =  3,  ffi(ei)  =  A,  and  ^2(64)  =  2.  WTien  we 
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want  to  refer  to  the  jet  of  all  sj-mbols  appearing  as  letters  in  a  particiilar  word,  we  shall  use 
an  unsubscripted  a.  Thus  we  would  have  <r{h)  =  (1,  >1,3}  and  <T(e3)  =  {3}  in  the  example 
above. 

Because  our  system  of  notation  suppresses  all  predicate  names,  it  is  not  powerful  enough 
to  directly  represent  a  linear  rule  such  as 

t(XiX2X3X4)  :  -  t(XiAAX2)  *  f(X3BA)  A  g(BA)  A  h(X4X2) 

that  has  multiple  EDB  predicate  names  1,  g,  and  h.  However,  we  can  easily  prove  that 
deciding  uniform  boundedness  for  linear  rules  with  multiple  EDB  predicates  allowed  is 
computationally  no  harder  than  deciding  the  question  for  linear  sirups  as  we  have  de¬ 
fined  them  above,  where  we  have  implicitly  assumed  them  to  involve  at  most  one  (possibly 
multiply-occurring)  EDB  predicate. 

Lemma  4.4  There  is  a  polynsmitd-tiTne  algorithm  that,  given  any  linear  recursive  Datalog 
rule  r,  returns  a  linear  Datalog  rule  r'  with  at  most  one  (possibly  multiply-occurring)  EDB 
predicate  such  that  r  is  uniformly  bounded  if  and  only  if  r'  u  uniformly  hounded. 

Proof  Suppose  r  is  a  linear  recursive  Datalog  rule  with  occurrences  of  different  EDB 
predicate  names  Each  predicate  name  L  may  occur  just  once,  or  alternatively 

several  times  as  the  name  of  an  EDB  subgoal  in  r.  Suppose  the  arities  of  are 

ai , . . . ;  a„  respectively,  and  suppose  that  the  the  arity  of  the  recursive  subgoal  in  r  is  t.  Let 
«  =s  1  -f  max(ai).  The  rule  r'  will  have  a  recursive  subgoal  of  arity  t  -f  s  and  a  single  EDB 
predicate  name  e  of  arity  o.  The  EDB  subgoals  of  r'  will  be  in  one-to-one  correspondence 
with  those  of  r,  and  are  constructed  from  them  as  follows.  First,  let  X{^ , . . . ,  A'f,  be  new 
(distinguished)  variable  symbols.  Suppose  /  is  some  EDB  subgoal  of  r,  and  suppose  the 
predicate  name  of  /  it  fi.  Then  to  construct  the  corresponding  EDB  subgoal  e  in  r\  we  let 
the  first  Oi  variables  of  e  of  agree  with  those  of  /,  and  then  pad  the  remaining  o  —  Oj  >  0 
positions  with  the  special  distinguished  variable  To  construct  the  recursive  subgoal 
of  r'  from  that  of  r,  we  copy  the  t  variables  of  the  recursive  subgoal  of  r  into  the  first  t 
positions  of  the  r'  recursive  subgoal,  and  then  fill  out  the  remaining  s  positions  with  the 
distinguished  variables  Jff,,. . .,  Jff,,  respectively.  The  head  of  the  new  rule  r\  (of  course, 
also  of  arity  t  -f  s),  is  similarly  copied  from  the  head  of  r  with  the  variables  A'f,, . .  • ,  A'f, 
occupying  the  last  s  positions. 

To  verify  that  the  reduction  given  above  is  a  correct  one,  it  suffices  to  check  that  for 
every  containment  mapping  between  two  recursive  expansions  of  r  there  is  a  containment 
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mapping  bttweM,  the  t«:<,  eonasponding  recur.i.t  axpaniioni  of  a',  „d  convmdv  But 
bafom  continuing,  d,  thtdl  Br,t  ttatplp  gi„  lui  cample  of  the  reduction. 

Example  4.5  V  e  return  to  rule 

t(X:AAX2)  ft  f(23BA)  4  g(BA)  t  h(X422). 

Iread,.  give.  .bo.e  «  »  exmnple  of  a  rule  that  h«  multiple  EDB  predicate  name.,  W 
fornung  the  rule  as  suggested  by  Lemma  4.4,  we  obtain  the  rule 

t(i.XrirX,i,I.in)  :  -  .(x.uxrx,x,xa)  a  .(,,3.1,)  a  a  .(i.ir„a). 


Firs.  ao7  T  I  "  "•'  "S"'  “ 

hi  r  V  “f  ““  <  •!»  Cue.)  dittinguitbed  enable.  A',  each 

the  idctitj  ntbgoal  .„b.titution  (JglA-,.)^  appbed  to  them  .-ben  r'  i.  rectjsively 

.TcTba  T 

t^r  b  i»  the  correrponding  recur.i,e  eipatuion  of  r,  except 

e  ha.  a  ...  of  paddmg  di..i„gui.b.d  variable.  Ag  occurring  in  it.  U..  .  -  a,,  pcitiaa, 
^au«  tb...  padding  .junboh  remairt  uncbnnini  under  reeurne  expanrio.  of  e'  and 
Wme  tbelind  (rigbtmo..)  padding  nrtabl.  i.  alway.  in  one-to-one  co.,..pond«.c.'wi.b 
e  origin  nme,  Ag  on  tbi.  .ubgoal  in  r,  every  containment  mapping  d  between  recurrive 
expannon.  of  ,  carrie.  over  direcUy  to  a  correrponding  contrunmoi.  mapping  p-  betwem, 
conerptm^g  rcurtive  expanrion.  of  r-.  and  converrdy.  Of  courm,  di.  only  differmce 
betwen.  tb...  two  mapping,  i.  tta.  ^(Aj)  =  Ag  i.  foreni  in  fli.  latter  mapping 


4.2  The  substitution  grtibph 

Previous  author.  [N.ug86.J.  (loanSS],  [JANSr)  have  ured  vmiou.  necbmnnn.  to  male  the 
proc.  by  which  a  Imear  recurrive  Da. Jog  rule  i.  recurrively  expeded  maUiematicJly 
preene,  and  m  the  premn.  then,  we  hae  premated  ye.  miotber  meUiod,  which  we  c  Jled 
flmtaette  expansion.”  However,  in  the  present  context  we  Jiall  prefer  tim  Witution 
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graphs  of  [JAN87],  although  our  notation  will  differ  slightly  from  that  paper.  Basically, 
a  substitution  graph  is  just  a  simple  way  of  describing  the  transformation  structure  of  the 
subgoal  substitutions  of  Section  2.2.3,  making  it  easier  analyze. 

More  formally,  let  Q  =  (A,  S)  be  a  given  linear  sirup  of  arity  n  with  a  set  of  k  EDB 
subgoals  f  =  {ei...  .,e*}.  The  substitution  graph  S  =  S{Q)  has  vertices  j  <t(A),  and 
a  directed  edge ,  <rj(A))  for  each  i  €  Pn-  In  other  words,  there  is  an  arc  from  each  integer 
i  to  the  symbol  appearing  in  the  ith  position  of  the  recursive  subgoal  of  the  rule. 

For  example,  consider  the  following  linear  sirup  Q  of  arity  13  (taken  from  [JANS?]): 

A22362(10)77(11)7B(11)(13)(15)  1.4459(12)5(14)8.  (4.1) 

The  sirup  Q  has  one  EDB  subgoal  ej  =  1A459(12)5(14)8,  and  a  15-ary  recursive  subgoal 
A.  (We  have  added  parenthes'  .o  our  representations  of  A  and  ej  only  in  order  to  allow 
symbols  from  Dis  which  exceed  9  to  be  vrritten  unambiguously).  The  substitution  graph 

5(2)  has  vertex  set  Pis  u  {A,  5}  and  the  15  directed  edges  represented  in  the  foUow'mg 
diagram. 


4  5 

\  / 

3  6 

\y 

2 

o 

We  are  now  almost  ready  to  describe  how  a  general  linear  sirup  2  =  (h,  S)  is  recursively 
expanded  into  an  infinite  sequence  of  conjunctive  queries  Q  =  2o,Gi,2a,23,...  whose 
union  specifies  the  relation  the  rule  defines.  We  need  a  preliminary  definition.  For  every  v 
in  V  =  P„  u  W,  we  define  its  successor  symbol  j(»)  as  follows.  First,  if  v  is  an  element  of 
Pn,  then  j(i»)  is  just  <t,(A);  i.e.,  we  simply  follow  the  unique  directed  edge  (v,  <r,(A)) 
out  of  V  in  the  substitution  graph  5  to  find  s(t;).  Alternatively,  we  may  have  v  €  W.  In 
this  case,  we  make  the  preliminary  definition  that  A  ss  A"  for  every  symbol  A  €  W,  and 


/ 
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then  inductively  define  the  successor  symbol  j(f*)  =  for  every  integer  i  >  0.  Thus  for 
example  j(/l)  =  3(^®)  =  A^,  and  we  have 

The  process  by  which  the  rule  Q  is  recursively  expanded  is  now  simple  to  describe.  To 
construct  Ci+i  from  2«,  we  first  replace  every  symbol  v  occurring  in  C,  by  its  successor 
symbol  s{v).  In  this  way  we  obtain  the  new  recursive  subgoal  for  Ci.j,  and  a  subset 
£'  of  its  £DB  subgoals.  Finally,  to  obtaiin  ,  we  adjoin  to  £'  the  original  EDB  subgoals 
£  of  2,  and  we  are  finished. 

To  illustrate  the  technique,  we  return  to  the  transitive  closure  rule  Q 

lA  A2, 


with  substitution  graph 

o 

1  2 - A 


We  can  write  the  infinite  sequence  of  conjunctive  queries  Q  =  2o<  Si>  fisi  Qa, ...  as  an 
infinite  triangular  query  tableau 


Qa 

X®2 

Qi 

li4* 

A^2 

0.2 

li4* 

i4»A* 

A^A^ 

A®2 

Ca 

lA* 

A^A^ 

am* 

A*  A®  A®2 

where  each  row  is  obtained  from  the  previous  one  by  first  substituting  s(v)  for  each  variable 
in  every  word,  and  then  adding  the  single  EDB  subgoal  ei  =  A!*2  onto  the  right*hand  end. 
For  a  second  example,  take  Naughton’s  rule 


1.43  .42  i43  33  32, 
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with  substitution  graph 


2 - ►  A 


The  rule  has  arity  n  =  3  and  it  =  4  EDB  subgoals.  The  query  tableau  for  Naughton’s  rule 
begins 

Co  1j4°3  i4''2  i4''3  33  32 

Cl  li4‘3  33  3^4°  .A‘‘2  A^’Z  33  32 

Cl  lA^Z  A^A^  A^Z  33  3^‘  AU°  .4'3  33  3^°  A^2  A'^Z  33  32 

Now  suppose  C  =  {h,  S)  is  an  arbitrary  linear  sirup  of  arity  n  with  Ir  >  1  EDB  subgoals. 
(Because  a  sirup  with  no  EDB  subgoals  is  always  uniformly  bounded,  the  case  of  1:  =  0 
will  concern  us  no  longer).  We  need  to  introduce  two  more  notations  in  order  to  refer  to 
particular  words  inside  a  querj'  tableau.  First,  we  shall  use  the  (previously  introduced) 
notation  h'  to  refer  to  the  leftmost  word  of  row  i  (i.e.  the  recursive  subgoal  in  the  first 
column  of  row  Qj).  Second,  we  will  use  the  notation  Ci(t,  X)  to  refer  to  the  EDB-subgoal 
word  in  column  1  +  (t  -  t)ib  +  A  of  row  i.  )ATien  using  the  latter  notation,  we  shall  always 

be  dealing  with  arguments  t  and  A  that  satisfy  0  <  t  <  t,  and  1  <  A  <  i.  If  either  condition 

fails,  C{(t,  A)  is  taken  as  undefined. 

The  general  sth  row  C,  of  the  query  tableau  of  a  linear  sirup  Q  with  arity  n  and  fc  >  1 
EDB  subgoals  therefore  consists  of  1  +  k{i  +  1)  words  written  left-to-right  in  the  following 
order: 

y  gi(t,  1) •  •  •  g.(t, k)  Qi(i  -  1, 1)  •  •  •  Qi{i  -  1, t) •  •  •gi(0.1)  -  g<(0,fe)- 
h  words  fc  words  fc  words 

jf  g  _  is  such  a  subgoal  word,  we  shall  also  occasionally  refer  to  1  as  the  Uvalue 

of  9,  and  we  shall  call  A  the  X-valite  of  q. 

Of  course,  we  can  also  think  of  the  ith-row  recursive  expansion  Qi  as  itself  defining  a 
linear  sirup  £<  =  (h',S*),  where 

r  =  UCi(t,A). 

I, A 
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We  shall  write  Q°°  for  the  infinite  set  {Q  =  Qo,  Qi,  Qj,  Q3, . . .}  of  aU  recursive  expan¬ 
sions  of  Q. 

As  examples  of  these  notations,  in  the  query  tableau  conesponding  to  Naugbton’s  rule 
we  have  h*  =  1A*3,  Cj(0,2)  =  A°3,  22(2,2)  =  A*3,  and  2i(l,4)  =  3A°. 

Next,  we  present  some  technical  lemmas  relating  to  query  tableaus  and  recursive  expan¬ 
sion  of  a  given  linear  sirup  Lemma  4.6  is  in  part  merely  a  verification  of  the  correctness 
of  the  recursive  expansion  procedure  as  we  have  outlined  it  above;  still,  we  shall  see  that  it 
has  some  far-reaching  consequences  in  the  study  of  uniform  boundedness. 

Lemma  4.6  Ift  <  i  then  2i+i(t,A)  =  Qi(t,A). 

Proof  We  can  prove  the  result  by  induction  on  t.  'WTien  t  =  0,  the  result  is  immediate 
because  2i+i(0,A)  =  Ci(C,A)  by  the  definition  of  the  rectirsive  expansion  process.  Now 
assume  the  result  to  be  true  for  all  relevant  values  of  t  and  A  provided  that  t  is  strictly  less 
than  some  positive  integer  j,  and  consider  the  case  of  t  =  j  >  0.  The  inductive  hvpothesis 
informs  us  that  the  two  words  Qi{j  -  1,A)  and  “  1»^)  well-defined  and  equal; 

moreover,  the  recursive  expansion  process  tells  us  that  in  order  to  construct  the  words 
C»+i(j,A)  and  Ci(i,A),  we  apply  the  successor  function  s  to  the  symbols  of  Qi{j  -  1,A) 
and  1,A),  respectively.  Since  each  successor  symbol  is  defined  uniquely,  we  have 

A)  =  2t(iiA),  as  required,  proving  the  inductive  step  and  completing  the  proof. 


Lemma  4.7  There  is  a  positive  integer  p  depending  only  on  Q  such  that  if  i  >  n  then  the 
foUoxoing  implications  are  valid: 

J.  IftTjih})  is  a  distinguished  variable^  then  (Tj{h}'^^)  = 

2.  If  ^jih*)  ^  is  a  nondistinguished  variable,  then 

Lemma  4.7  is  the  central  result  of  [JAN87]  (Theorem  1,  page  337),  and  we  shall  not 
reprove  it  here.  The  smallest  positive  integer  p  satisfying  the  conditions  of  Lemma  4.7  is 
called  the  period  of  2*  When  the  substitution  graph  5  of  2  contains  at  least  one  cycle, 
then  p  is  the  least  common  multiple  of  the  cycle  lengths  in  5.  If  the  substitution  graph  of 
2  contains  no  cycles,  then  taking  p  =  1  will  satisfy  the  conditions  of  the  lemma. 

Lemma  4.8,  below,  is  also  implicit  in  the  results  of  [JAN87],  and  we  shall  omit  a  formal 
proof  of  this  result  as  well.  Intuitively,  the  result  expresses  the  idea  that  the  variable 
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patterns  of  the  EDB  subgoals  in  recursive  expansions  of  a  linear  sirup  Q  of  arity  n  also 
follow  easily  described  patterns  after  the  nth  recursive  expansion  Cn- 

Lemma  4.8  Let  Q  be  a  linear  sirup  of  arity  n,  and  suppose  the  period  of  Q  is  p.  If  i  >  n 
and  t  >  n,  then  the  following  implications  are  valid. 

1.  If'fjiQt{t,h))  is  a  distinguished  variable,  tAen  +  p,  A))  =  (T^(Ci(^,A)). 

2-  If  (rj{Qi{t,X))  =  »5  a  nondistinguished  variable,  then  <rj{Qi.^p{t  -p,  A))  =  .4*"^. 

To  study  containment  mappings  <p  between  the  various  recursive  expansions  we  need 
to  translate  our  earlier  description  of  these  mappings  (conditions  1-4  from  Section  2.2.1, 
above)  into  the  present  system  of  notation. 

Let  2  =  (A,  £)  be  a  linear  sirup  of  arity  n  with  k  EDB  subgoals.  Then  a  function 
d>  '•  Qi  Qj  vrill  be  a  containment  mapping  if  it  satisfies  the  following  properties: 

1.  4»(A*)  =  hi. 

2.  ^(Q.(t,A))  €  £i  for  all  f  and  A. 

3.  If  <T((A')  is  a  distinguished  s-ariable,  then  et{^{h*))  =  (T({hi)  is  the  same  distinguished 
variable. 

4.  If  tri{Qt{t,  A))  is  a  distinguished  variable,  then  <ri{(h{Qi{t,  A)))  is  the  same  distinguished 
variable. 

5.  If  <r|,(C<(ti,Ai))  =  <rj,(Ci(t2.Aa))  then  ff/,(^(Q<(ti,Ai)))  =  <r/,(^C<(t2,  Aj))). 

6.  If  (T,,(Ci(<t,A,))  =  then  <r,,(^(Ci(fj,Aj)))  =  «r,,(^(A‘)). 

Here,  we  can  see  that  Conditions  1  and  2  above  are  just  restatements  of  our  earlier 
Conditions  1  and  2  from  Section  2.2.1,  while  the  present  Conditions  3  and  4  jointly  restate 
Condition  3  firom  Section  2.2.1.  Finally,  Conditions  5  and  6  above  restate  Condition  4  from 
Section  2.2.1. 

A  containment  mapping  Qi—*  Qj  is  called  nontrivial  Hi  ji  j.  We  have  already  seen 
that  the  relation  >  on  given  by  Ci  >  if  and  only  if  there  is  a  containment  mapping 

Qi  —  Qj  is  reflexive  and  transitive. 

Recall  that  Q  is  called  uniformly  bounded  if  there  is  a  constant  N  such  that  for  all 
3  ^  we  have  fij  ^  Cj  for  some  i  <  N. 
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Also,  recall  that  corresponding  to  every  containment  mapping  ^  :  C,  —  C,  there  is  a 
unique  derived  mapping 

i  :  it(Qx)  —  <r{Q}) 

which  is  a  mapping  between  the  variables  occurring  in  C,  and  those  occurring  in  C;  and  is 
given  by  the  two  rules  that 

for  every  / 
and 

=»  ^(<T,(Ci(ti,Ai)))  =  ei{Qi{t3,Xi))  for  every  L 

That  the  mapping  ^  is  well-defined  by  these  two  rules  is  a  simple  consequence  of  the  axioms 
for  a  containment  mapping. 

There  are  several  technical  results  which  we  shall  need  to  verify  before  continuing: 

Lemma  4.9  Let  Q  be  a  linear  sirup  of  arity  n.  If  is  a  nondistinguished  variable  occur- 
ring  in  Qi{t,X),  then  0  <  t  —  I  <  n. 

Proof  We  prove  the  result  by  induction  on  i.  First,  note  that  if  s  =  0,  then  t  =  I  =  0 
and  the  result  is  trivially  true.  Inductively,  suppose  that  the  lemma  is  valid  for  all  values 
f  strictly  less  than  some  fixed  integer  M,  and  consider  the  case  i  =  M.  First,  note  that 
if  /  >  0,  then  i  and  t  are  both  >  1  and  the  EDB  subgoal  -  1,A)  must  contain  the 

nondistinguished  variable  A*"*  in  order  for  Qi{t,  A)  to  contain  A*.  The  induction  hypothesis 
then  implies  0  <  (t  -  1)  -  (/  -  1)  <  n;  in  other  words,  0  <  « -  /  <  n,  and  we  are  finished. 

Alternatively,  we  may  have  /  =  0.  Suppose  that  ^j(Ci(t,A))  =  A*  =  A®.  In  this  case, 
if  it  is  not  already  the  case  that  t  =  0,  then  by  the  definition  of  the  recursive  expansion 
process,  there  are  exactly  t  different  nonpersistent  variables  -  1,  A)),7j(Si.2(t  - 

2,  A)), ... ,  A))  “above"  <r,(Ci(t ,  A))  in  the  query  tableau  for  Q.  Since  n  is  a  upper 

bound  on  the  number  cf  such  nonpersistent  variables,  we  have  0  <  t  <  n,  or  in  other  words, 
0<t-/<n,  as  required.  I 

A  simple  corollary  of  the  previous  Lemma  is 


Lemma  4.10  Suppose  A**  €  and  5**  €  <r(2i(faiAj)).  Then  l(tt  -  (2)  -  (fj  - 

/j)l  <  n. 
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Proof  Applying  Lemma  4.9,  we  find  that  0  <  -  /j  <  n  and  0  <  ~  /j  <  n. 

Subtracting,  we  conclude  that  \{t\  -  fa)  -  (/i  -  fj)!  <  n,  as  required.  I 

A  second  useful  corollary  of  Lemma  4.9  is 

Lemma  4.11  Suppose  (r(2i(fx,Ax))  n  (r(2i(f2,  Aa))  confatns  some  nonduftn^tsAei  tan* 
able  Then  if;  -  fa!  <  n. 

Proof  We  apply  the  previous  lemma  with  A  =  J?  and  =  /a  =  L  ■ 

Finally,  it  is  useful  to  have  a  characterization  of  the  exponents  that  can  appear  on  the 
nondistinguished  variables  in  the  recursive  subgoal  h}  of  a  recursive  expansion  Qii 

Lemma  4.12  Suppose  occurs  in  h\  the  recursive  subgod  of  the  iih  recursive  expansion 
Qi  of  a  linear  sirup  Q  of  arity  n.  Then  max(0,  <  -  n)  <  /  <  i. 

Proof  We  may  prove  the  lemma  just  as  we  proved  Lemma  4.9,  above,  by  induction  on 
t.  First,  note  that  if  t  -  0,  then  /  0  and  the  lemma  is  trivially  true.  Inductively,  suppose 

that  the  lemma  is  valid  for  ail  values  t  strictly  less  than  some  fixed  integer  Jlf ,  and  consider 
the  case  t  =  Af .  If  /  >  0,  then  t  >  1  and  we  see  that  h*^^  must  contain  the  nondistingtxished 
variable  in  order  for  A*  to  contain  A^  The  induction  hypothesis  then  implies  that 
max(0,  i~l-“n)</~l<i-l,  which  implies  max(0,  t  -  n)  <  /  <  i,  as  required. 

Alternatively,  we  may  have  /  0.  Suppose  that  trj{h^)  s  A^  =  A^.  In  this  case,  if 

it  is  not  already  the  case  that  t  0,  then  by  the  definition  of  the  recursive  expansion 
process,  there  are  exactly  i  different  nonpersistent  variables 

*'above’*  in  the  query  tableau  for  Q-  Since  n  is  a  upper  bound  on  the  number  of 

such  nonpersistent  variables,  we  have  t  <  n,  or  in  other  words,  because  /  0,  we  have 

maz(0,  t  n)  <  I  <  t,  as  required.  I 

4.3  Containment  depth 

At  last  we  are  in  a  position  to  be  able  to  prove  the  result  already  alluded  to  above: 

Theorem  4.13  There  is  a  constant  K  depending  only  on  Q  such  that  if  Qi  >  Qj  for  some 
j  >i>l,  then  Qi  v  Qp  for  some  j'  satisfying  i<  j’  <  Ki. 
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Proof  We  shall  see  that  the  constant  K  may  be  taken  as  roughly  proportional  to  the 
product  pnk  of  the  period  p  of  Q,  its  recursive  subgoal  arity  n  and  the  number  k  of  its 
EDB  subgoals;  more  precisely,  we  claim  that  the  choice  K  =  ipnk  suffices.  Our  method  is 
essentially  a  “reverse  pumping  argument”  as  has  been  used  for  example  in  [Cosm88]. 

Fix  I,  and  consider  the  smallest  >  i  such  that  Q,  >  C/.  To  prove  the  theorem,  we 
shall  show  that  if  C«  Cj  j  >  then  j  ^  j'\  i.e.,  j  is  not  minimal. 

So  suppose  that  and  j  >  Ki.  Let  0  :  Ci  —  Qj  be  a  containment  mapping. 

Qj  is  represented  by  the  jih  row  of  the  query  tableau  for  Q.  This  row  is 

Ci(i,  1)  •  •  •  C;(j,  ifc)  -  1. 1)  •  •  •  c>(i  “  1.  *)  •  •  •  2;(0. 1)  •  •  •  *). 

k  words  k  words  fc  words 

and  it  consists  of  the  recursive  subgoal  and  k{j  +  1)  EDB  subgoals.  We  shall  focus  on 
the  first  (i  -  n)fc  EDB  subgoals  from  the  left  in  Qj,  and  shall  think  of  the  leftmost  of  these 
subgoals  as  broken  into  T  =  [{j  -  n)/(pn)J  “blocks”  of  pnk  EDB  subgoals,  apiece.  Note 
that 

j -n>  Ki-n=:  Apnki  -  n  =  (4pfc  -  l)n  >  3np, 

so  we  are  definitely  looking  at  a  nonempty  collection  of  subgoals  here.  Now  Q;  has  only 
i(f  +  1)  EDB  subgoals,  and  because 

j  >  Ki 
=  Apnki 
>  n  +  pnfc(»  +  1), 


we  conclude  that 

T  >  (t  +  l)t. 


By  the  pigeonhole  principle,  there  is  some  block  of  Qj  into  which  ^  maps  no  EDB  subgoal 
a(t,A).  Let 


Qj(M,  1)  •  •  •  Qi(K  ky  -  •  g,( Af  -  pn  +  1, 1)  •  y  g,(Af  -pn  +  1,  k) 
k  words  k  words 


be  the  first  (i.e.,  leftmost:  choose  the  maximal  such  M)  block  of  Qj  into  which  ^  maps  no 
EDB  lubgoal  of  Qi.  Note  that  the  value  M  falls  in  the  closed  interval  \j  -  [(;  -  n)/(pn)J  + 
l,y],  and  in  particular  we  have 


M>3- 


pn 


pn 


+  1. 
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claim  that  in  fact  M  >  n,  also.  To  see  this,  we  can  write 

pn*  <  pn*  -  n  -  n(p  -  1) 

<  (pn-l);-i-n(p-l) 

=  Pnj  -  U  -  n) -r  pn, 

and  dividing  both  sides  of  this  inequality  by  pn  we  obtain 

n  <  i  -  +  1  <  M, 

pn 

as  claimed. 

To  finish  the  proof,  we  shall  show  that  there  is  a  second  containment  mapping 

that  is  constnictible  from  4>  and  A/  as  follows.  First,  we  define  ^i(h‘)  =  as  we 

tnust.  Then,  to  define  for  an  EDB  subgoal  of  Qi,  we  first  examine  the  image 

By  our  arguments  above,  we  must  have  either  tj  >  A/  or 
*2  <  Af  -  pn  -f  1;  if  the  former  is  the  case,  then  we  define  ^(Qi(ti,Xi))  =  Qj-p{t2  -  p,  A2); 
and  if  the  latter,  then  we  define  ^i(Q<(fj,Ai))  =  Q,-p(t2,A2). 

Next,  we  check  that  the  t*values  for  Qj-p  we  have  just  defined  lie  in  the  proper  ranges. 
Consider  first  the  case  of  <2  >  M.  We  need  to  check  that  0  <  «2  -  p  <  j  -  p.  W'e  easily 
see  that  t2—p<  j—p because  <2  $  i» by  assumption.  To  verify  that  0  <  ts ~P.  it  i*  enough 
to  check  that  p  <  Af,  which  we  can  do  as  follows.  We  know  that  the  first  block  of  Qj  not 
ms-pped  into  by  ^  fan  be  the  (1  +  (t  +  l)k)th  block  firom  the  left,  at  the  latest.  Therefore 
M  >  j  —  {i  +  l)fc;  we  need  to  check  that  j  —  (t  +  1)4  >  p  to  finish.  Now 

i  -  (» +  1)4  >  4pn4t  -  (t  -  1)4 

>  24t(pn) 

>  P, 


as  required. 

Now  consider  the  second  case  of  the  definition  above,  i.e.  the  case  of  <2  <  Af  -  pn  t  1. 
We  need  to  check  in  this  case  that  0  <  t2  <  3  -  p.  We  already  know  that  0  <  <2  by 
assumption,  so  the  problem  reduces  to  showing  that  t2  <  3  -  p.  To  see  this,  note  that  if 
t2  <  M  -  pn  -r  1,  then  there  are  at  least  pn  blocks  to  the  left  of  the  t2  block  in  Qj,  none 
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of  which  is  mapped  into  by  <ft.  Therefore  i  >  tj  ^  pn  >  it  p,  *o  **  have  tj  <  j  —  p, 
required. 

All  that  remains  to  done  is  to  check  that  <f>i  is  indeed  a  containment  mapping;  we 
must  verify  the  six  points  of  the  definition  in  turn.  Because  these  points  are  basically  just 
mechanical  verifications,  we  shall  place  them  in  the  appendix  for  the  sake  of  clarity. 


If  additionally  one  could  give  an  explicit  bound  in  terms  of  r  on  the  maximal  depth 
i  at  which  the  first  nontrivial  containment  r*  ^  occurs  for  a  bounded  rule  t  in  the 
previous  theorem,  then  we  would  easily  obtain  an  alternative  proof  of  \ardi  s  result  that 
the  boundedness  problem  is  decidable,  but  such  a  proof  has  thus  far  eluded  us. 

4.4  Cuts 

Let  Q  =  (h,£)  be  a  linear  sirup  of  arity  n  with  k  EDB  subgoals,  and  let  Ci  =  be 

a  recursive  expansion  of  The  linking  graph  Qi  is  an  undirected  graph  on  the  vertices 
h*  u  f*,  with  an  edge  between  two  words  if  they  share  a  nondistinguished  variable.  The 
connected  component  of  Qi  to  which  h*  belongs  is  called  the  head  chain  ot  If  the  number 
of  vertices  belonging  to  the  head  chain  of  is  unbounded  as  i  increases,  then  we  say  that 
Q  has  an  unbounded  head  chain.  The  foUow’ing  result  is  &om  [Naug86a],  although  we  have 
slightly  changed  the  terminology  used  in  that  paper: ' 

Lemma  4.14  Let  Q  be  a  linear  single  rule  program.  If  Q  is  not  uniformly  bounded,  then 
Q  has  an  unbounded  head  chain. 

Naughton  and  Sagiv  (Naug86a],  [NaSa87]  and  loannidis  [Ioan85]  give  efficient  (linear- 
time)  algorithms  for  deciding  whether  a  linear  sirup  C  has  an  unbounded  head  chain. 
Unfortunately,  a  rule  may  have  an  unbounded  head  chain  and  still  be  uniformly  bounded. 
Naughton’s  rule, 

1A3  A2  AZ  33  32, 

already  mentioned  above,  is  perhaps  the  simplest  example  of  such  a  rule.  These  results  led 
Naughton  and  Sagiv  [NaSa87]  to  study  sufficient  conditions  for  a  rule  with  an  unbounded 
head  to  be  in  fact  unbounded.  They  present  four  different  polynomial-time  testable 
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conditions,  each  sufficient  for  a  rule  with  an  unbounded  head  chain  rule  to  be  unbounded. 
Here,  we  generalize  these  results  to  give  a  single  polynomial-time  testable  sufficient  condi¬ 
tion  for  the  unboundedness  of  a  rule  with  an  unbounded  head  chain  that  simultaneously 
subsumes  all  four  classes  identified  by  Naughton  and  Sagiv.  ^^'e  shall  caU  our  condition  the 
cut  condition.  Along  the  way,  we  shall  need  to  prove  two  preparatory  lemmas  that  closely 
parallel  lemmas  due  to  Naughton  and  Sagiv  [NaSa87]— in  particular,  our  Lemmas  4.16 
and  4.17  correspond  to  Naughton  and  Sagiv’s  Lemmas  4.2  and  4.5,  respectively.  Our  alter¬ 
native  proofs  of  these  two  lemmas,  however,  together  with  our  more  detailed  notation,  will 
allow  us  to  deduce  some  additional  information  that  will  lead  to  the  cut  condition. 

First,  we  must  continue  with  some  more  definitions.  We  shall  call  a  distinguished  variable 
d  €  ‘P„  of  Q  persistent  if  d  belongs  to  some  (possibly  trivial)  directed  cy  cle  in  the  substitution 
graph  5  of  Q.  To  state  it  another  way,  d  is  persistent  if  by  following  successor  symbols  away 
from  d  in  5  we  find  that  s*(d)  =  d  for  some  t  >  0.  Otherwise,  d  will  be  called  nonpersisient. 
(The  two  definitions  follow  those  of  [Naug86a]).  If  a  distinguished  variable  d  is  nonpersistent, 
then  there  are  two  possible  subcases:  either  there  is  a  unique  positive  integer  t  <  n  and 
a  unique  nondistinguished  variable  A  ^  14  such  that  s*(d)  —  A  ~  A®,  in  w’hich  case  we 
shall  say  that  the  variable  d  belongs  to  A,  and  that  d  has  height  t;  or  alternatively,  the 
nonpersistent  variable  d  must  belong  to  a  component  of  the  substitution  graph  5  of  C  that 
contains  a  cycle  (but  d  is  not  actually  in  the  cycle),  in  which  case  we  shall  say  that  d  is  a 
stem  variable. 

For  example,  in  the  rule  4.1,  above,  the  distinguished  variables  2,  7,  10,  11,  and  13 
are  persistent,  while  the  remaining  distinguished  variables  1,  3,  4,  5,  6,  8,  9,  12, 13,  and 
14  are  nonpersistent.  Amongst  the  nonpersistent  variables,  i  and  12  belong  to  A  and  B 
respectively,  while  3,  4,  5,  6,  8,  9, 13,  and  14  are  stem.  The  heights  of  the  variables  1  and 
12  are  both  equal  to  one. 

If  a  nonpersistent  variable  d  belongs  to  A  and  the  height  I  of  d  is  maximal  amongst 
all  nonpersistent  variables  of  Q  belonging  to  A,  then  we  call  d  a  leadety  or  alternatively  an 
A-leader  when  we  wish  to  remind  ourselves  to  which  nondistinguished  variable  the  variable 
d  belongs.  We  shall  also  say  that  the  variable  A  itself  belongs  to  A,  and  we  define  the  height 
t  .of  A  to  be  zero. 

Next,  let  e,  he  an  EDB  subgoal  of  the  linear  sirup  C,  and  let  d  be  some  fixed  distinguished 
variable  of  fi.  Then  we  shall  say  that  e<  is  A-leader^containing  if  <r,(ei)  is  an  A-leader  for 
some  s.  Finally,  we  shall  say  that  a  second  EDB  subgoal  Cj  is  a  (d,  A)-ciit  of  Ci  if  the 
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following  properties  hold: 

1.  ti  is  X-leader-containing. 

2.  d  is  either  a  stem  \’ariable  or  a  persistent  variable. 

3.  In  some  position  s  of  ej  in  which  an  .A>leader  appears,  tj  has  the  >'ariable  d. 

4.  If  ff,(ei)  is  persistent  or  stem,  then  <r,(ei)  and  <T,(ej)  are  in  the  same  component  of 
the  substitution  graph  of  Q  (so  in  particular  v,{ej)  is  either  persistent  or  stem). 

5.  If  erf{ei)  belongs  to  some  nondistinguished  variable  B  but  is  not  a  S-leader,  then 
ff,(e^)  belongs  to  some  nondistinguished  variable  C  (we  allow  A  =  B  and/or  B  =  C 
here). 

6.  If  ff«(e,)  is  an  j4-leader,  then  belongs  to  the  same  component  as  d  in  the 

substitution  graph  of  2- 

Is 

Notice,  in  particular,  that  no  restrictions  are  placed  on  leaders  that  belong  to  nondistin¬ 
guished  variables  other  than  A.  We  say  more  simply  that  Q  has  a  cut  if  is  a  (d,  ./4)-cut 
of  a  for  some  appropriate  values  d.  A,  i,  and  j. 

Our  main  goal  in  the  present  section  is  to  prove: 

Theorem  4.15  If  Q  is  uniformly  bounded  and  has  an  unbounded  head  chain,  then  Q  hri 
a  cut. 

We  shall  require  two  intermediate  results  (Lemmas  4.16  and  4.17)  before  we  prove  the 
theorem.  Before  we  continue,  it  is  useful  to  reexamine  Naughton's  rule  once  more  in  order 
to  get  some  intuition  for  the  eventual  proof  of  Theorem  4.15,  below. 

In  Naughton's  rule,  the  EDB  subgoal  es  =  >43  is  a  (3,  A)-cut  of  cj  =  A2.  We  can  think 
of  this  cut  as  arising  from  a  containment  mapping  in  the  following  way.  We  found  above 
that  the  query  tableau  for  Naughton’s  rule  begins; 

Co  1>1®3  A^2  A^Z  33  32 

Cj  1>4*3  >4*3  33  3>4»  A^2  >t®3  33  32 

Cj  M’3  A^A^  A^Z  33  3>4*  >4^®  A‘3  33  3A®  A®2  A®3  33  32 
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Co  1^‘’3  -^"2  ^"3  33  32 


Gj  M*3  A^3  33  3A*  A^3  33  34®  4"2  4®3  33  32 


Figure  4.1:  A  containment  mapping  ^  :  2i  2}  in  Naughton’s  rule. 

Exhaustively  checking  possible  mappings,  one  finds  that  there  is  no  containment  map* 
ping  ^  :  2o  -*  2ii  ^50  that  there  is  no  containment  mapping  Qo  -*  Cr-  However, 
there  is  a  containment  mapping  ^  :  Qi  — »  Qj,  as  indicated  by  the  arrows  in  Figtire  4.1. 

Several  points  about  the  mapping  ^  can  be  made: 

1.  There  is  a  nondistinguished  variable  of  2i  (here,  the  variable  A^)  that  ^  maps  to  a 
persistent  distinguished  variable  (here,  the  variable  3). 

2.  The  variable  is  on  the  head  chain  of  2i* 

3.  Looking  strught  above  the  leftmost  occurrence  of  A**  in  2i  in  the  top  row  of  the 
query  tableau,  we  encounter  the  EDB  subgoal  ei  s  A2,  which  contains  the  A>leader 
variable  2. 

4.  Looking  above  the  image  of  the  leftmost  A*^*contaming  subgoal  in  2i ,  (in  other  words, 
looking  above  the  subgoal  A’3  in  Ss),  we  encounter  the  EDB  subgoal  C)  =  A3,  which 
is  a  (3,A)>cut  of  ei. 

Basically,  the  idea  of  the  cut  condition  is  this:  the  above  derivation  of  a  cut  in  2o  a 
containment  mapping  ^:Qi  -*  Qi  can  be  generalized.  In  fact,  we  shall  see  that  whenever 
a  rnntMtiTnijnt  mapping  ^  :  Qi  Qj  (with  i  <  j)  exists  in  a  query  tableau,  then  we  can 
produce  a  cut  in  the  rule  2o  itself  by  "looking  above”  certain  subgoals  in  and  Qj. 


66 


CHAPTER  4.  SINGLETON  RULE  SETS 


We  return  now  to  the  preparatory  lemmas  mentioned  above. 

Our  first  preparatory  Lemma  asserts  that  when  Q  has  an  unbounded  head  chain,  there 
are  EDB  subgoals  that  contain  nonpersistent  variables  and  belong  to  the  head  chain  of  G; 
in  every  recursive  expansion  C;: 

Len*ma  4.16  Suppose  fi  is  o  linear  sirup  of  arity  n  with  k  EDB  subgoals,  and  suppose  Q 
has  an  unbounded  head  chain.  Let  Q,  be  an  arbitrary  recursive  expansion  of  Q.  Then  there 
is  an  EDB  subgod  e  -  Ci{f,  A)  of  Qi  such  that  the  following  conditions  are  met: 

1.  e  belongs  to  the  head  chain  of 

2.  e  contains  a  nonpersistent  variable. 

Proof  Suppose  to  the  contrary  that  Qi  is  a  recursive  expansion  of  Q  that  has  no 
nonpersistent *variable*containing  EOB  subgoals  on  its  head  chain.  First,  note  that  the 
head  chain  of  every  Qt  with  I  >  i  contains  at  least  as  many  EDB  subgoals  as  the  head 
chain  of  Qi,  because  if  Qi{t,X)  belongs  to  the  head  chain  of  Qi,  then  we  easUy  see  that 
Qi*a{t  +  a,  A)  belongs  to  the  head  chain  of  Gi+a •  Moreover,  because  such  a  Gi(t,  A)  contains 
no  nonpersistent  variables,  neither  does  Gi>a(^  +  A).  Now  because  C  has  an  unbounded 

head  ch^,  there  is  some  ;  >  i  such  that  the  recursive  expansion  Cj  has  strictly  more  EDB 
subgoals  on  its  head  chain  than  Gi  has  on  its  head  chain.  Choosing  the  minimal  such  j ,  we 
rlaim  that  Gj-1  must  have  a  nonpersistent  variable  on  its  head  chain,  a  contradiction. 

To  prove  the  claim,  let  qi  =  Qj(ti,Xi)  be  a  “new"  member  of  the  head  chain  of  (i.e. 
Cj(ti,Ai)  is  not  a  shift  Ci+«(t  +  ^lA)  of  some  subgoal  Qi(t,X)  on  the  head  chain  of  Qi, 
with  i  =  t  +  a).  We  may  also  assume  that  qi  is  chosen  so  that  it  shares  a  nondistinguished 
variable  A'  with  some  qj  =  Gi4^(<3  +  «>  Aj),  such  that  i  +  o  =  y  and  Gi(t2,  A])  is  on  the 
head  chain  of  Qi.  First,  note  that  I  must  be  sero,  for  if  1  >  0,  then  q[  =  Gj-iC^i  -  li  Ai)  is 
similarly  a  new  member  of  the  head  chain  of  Gj-ii  und  we  have  chosen  j  minimal.  So  /  =  0 
and  we  see  that  +  a  -  1,  Aj)  must  contain  a  nonpersistent  variable  in  order  for 

this  A®  to  arise  in  ft.  Because  this  Gi^t-a-iC^s  +  a  ~  It  Aj)  is  on  the  head  chain  of  Gi.4.a-ii 
we  have  our  contradiction.  I 

Next,  we  prove  that  if  a  rule  Q  with  an  unbounded  head  chain  is  bounded,  then  there 
must  be  a  containment  mapping  ^  :  Qi  Qj  between  two  recursive  expansions  of  Q 
with  i  <  j  such  that  the  derived  mapping  ^  maps  some  nondistinguished  variable  A'  to  a 
persistent  distinguished  variable  d: 
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Lemma  4.17  Suppose  Q  is  a  linear  simp  of  arity  n  with  k  EDB  subgoats,  suppose  Q  has 
an  unbounded  head  chain,  and  suppose  that  2  u  bounded.  Then  there  are  integers  i,  j ,  and 
I,  and  a  persistent  distinguished  variable  d  such  the  following  conditions  are  met: 

J.  There  is  a  containment  mapping  <h '•  Qi  -*  Qj  with  i  <  j. 

2.  ^{A^)  =  d  for  some  nondistinguished  variable  A*  of  Qj. 

3.  A*  occurs  in  an  EDB  predicate  on  the  head  chain  of 

Proof  Because  fi  is  bounded,  there  is  a  containment  mapping  <ff'  ■  Qi  —  Qy  for 
some  two  values  i  <  j'  (Theorem  4.1).  Also,  by  applying  the  splicing  lemma  repeatedly  if 
necessary,  we  can  asstime  that  i  >  n  without  any  loss  of  generality.  By  Lemma  4.15,  we  know 
that  the  head  chain  of  Qi  has  some  nonpersistent-variable*containing  subgoal  q  =  Qi{t,X) 
belonging  to  it.  Let  fc*  =  Jo,  9i,  ft.  •  •  • .  =  9  be  shortest  sequence  of  subgoals  of  Qi  that 

“connects”  q  to  the  head  chain  of  fi,-,  in  other  words,  for  each  pair  (9<.9i-i)  ^e  have  that 
9i  and  9i.fi  share  some  nondistinguished  variable,  and  P  is  chosen  as  minimal  with  respect 
to  sequences  that  connect  h’  to  q.  (Such  a  sequence  is  most  easily  thought  of  as  a  shortest 
path  between  h*  and  q  in  the  linking  graph  Qi  of  Qi).  Because  Ci  has  exactly  k{i  + 1)  EDB 
subgoals,  we  have  P  <  k{i  +  1)  here. 

Now  suppose  that  contrary  to  the  present  theorem  statement,  there  is  no  containment 
mapping  from  one  recursive  expansion  of  Q  to  another,  deeper  one  that  maps  a  nondis* 
tinguished  variable  to  a  persistent  variable  d.  We  would  like  to  obtain  a  contradiction.  In 
our  proof  of  Theorem  4.1,  we  saw  that  2i  V  Q^i  implies  that  Qi  >;  Qmi'-(m-i)t  for  every 
positive  integer  m.  In  particular,  consider  the  choice  m  =  4nk(t + 1),  let  j  =  mj'  —  (m  —  l)i, 
and  suppose  that  ^  :  Qi  Qj  is  an  associated  containment  mapping.  We  shall  show  that 
our  rombinH  hypotheses  imply  that  ^9)  cannot  contain  a  nonpersistent  variable  (as  it 
must,  if  ^  is  to  be  a  containment  mapping) — a  contradiction. 

In  fact,  we  shall  need  to  make  the  slightly  stronger  claim  that  the  t-value  of  ^(ft)  exceeds 
(P  +  2  -  i)n  for  every  s’  in  the  range  1  <  t  <  P.  Once  we  have  proved  the  claim,  we  shall 
know  in  particular  that  the  l-value  of  d>{qp)  ~  ^(9)  ••  greater  than  2n.  Because  no  EDB 
subgoal  with  a  t  value  greater  than  n  can  have  a  nonpersistent  variable  occurring  in  it,  we 
shall  have  our  contradiction. 

We  can  prove  the  elM*n  by  induction  on  s.  First,  because  h*  =  90  and  ft  share  a  nondis* 
tinguished  variable,  and  ^h*)  =  V  has  all  of  its  nondistinguished  variable’s  exponents  in 
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the  range  [j  -  n,  n]  (Lemma  4.12),  we  see  that  the  t- value  t  of  )  in  Qj  can  be  no  smaller 
than  j  -  n,  by  Lemma  4.9.  Now 

3  -  n  =  mj'  -  (m  -  1):  -  n 
=  ”»(/  -  t)  (i  -  n) 

>  4njk(i  +  l)(j'  -  t) 

>  (P4-l)n, 

so  t  >  (P  -!•  l)n,  as  required  in  the  base  case  of  our  claim. 

Inductively,  suppose  the  claim  is  \-alid  for  a  value  i  =  M  ~  I  <  P  and  consider  the  case 
of  t  +  1  =  Af .  We  know  that  qi  and  fi.,.)  share  some  nondistinguished  variable  .4'.  The 
induction  hypothesis  implies  that  ^{qi)  has  a  t- value  >  (P  +  2  -  »)n  >  n,  so  can¬ 
not  be  a  nonpersistent  distinguished  variable,  and  by  assumption  is  not  a  persistent 
distinguished  variable.  Therefore  =  B*  for  some  nondistinguished  variable  b  with 

exponent  a,  say;  in  other  words,  ^(9«)  and  share  a  nondistinguished  variable.  Ap¬ 

plying  Lemma  4.11,  we  find  that  the  condition  that  the  (-value  of  <t>{qi)  exceeds  (P  -f  2  -  t)n 
implies  that  the  (-value  o{4>{Qi+i)  can  be  no  smaller  than  (P-t-2-i)n-n  =  (P-*-2-(s-t-l))Ti, 
as  we  were  required  to  show.  This  completes  the  proof  of  the  claim,  and  of  the  lemma. 

I 

We  can  return  now  to  the  proof  of  Theorem  4.15: 

Proof  Let  2  be  a  linear  sirup  of  arity  n  with  k  EDB  subgoals,  and  suppose  Q  has  an 
unbounded  head  chain,  yet  is  bounded.  We  need  to  show  C  has  a  cut.  By  Lemma  4.17, 
there  are  integer  tuples  such  that  i  <  j,  and  we  have  a  containment  mapping 

Qi  -»  Qj,  where  ^(A')  =  d'  for  some  nondistinguished  variable  A  of  exponent  I  in 
and  d!  is  some  persistent  distmguished  variable.  By  applying  the  splicing  lemma  and  the 
transitive  property  of  containment  repeatedly  if  necessary,  we  can  assume  without  any  loss 
of  generality  that  the  tuple  satisfies  the  conditions  t  >  3n  and  j  >  4nk(t  -I- 1), 

and  that  -  <*,A))  =  ^Qj{j  -  a,  A))  for  all  o  in  the  range  0  <  o  <  n.  Now 

suppose  that  q  =  2t((i,Ai)  is  the  leftmost  EDB  subgoal  on  the  head  diain  of  Qi  that 
contains  a  variable  A'  such  that  ^(A')  =  (f  for  some  persistent  variable  d',  and  suppose 
that  ^(g)  =  ^(C»((i,Aj))  =  QjiU,Xi),  and  o,{q)  =  A'.  It  follows  that  (x  <  i  -  n  here. 

We  claim  that  the  EDB  subgoal  ex,  is  a  (d,  A)-cut  of  ex,  in  2,  where  d  =  w«(ex,)  defines 
d. 
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To  prove  the  claim,  we  must  verify  the  six  points  1-6  of  the  definition  of  a  cut,  above. 

To  verify  condition  1  for  a  cut,  suppose  that  <rr(q)  =  .4*,  as  above.  Then  we  claim 
that  tri.(eA, )  is  an  .4-leader.  Clearly  cr„(eA, )  belongs  to  .4,  by  the  definition  of  the  recursive 
expansion  process.  We  must  show  that  <r„(eA,)  has  maximal  height  amongst  variables  of  2 
belonging  to  A.  Every  .A-leader  must  occur  in  some  EDB  subgoal  of  Q,  for  otherwise  Q  is 
not  safe.  Suppose  then  that  ex  is  an  EDB  subgoal  of  Q  containing  an  .4-leader  (r*(eA),  and 
that  <rv(eA,)  has  a  strictly  smaUer  height  than  <r*(eA).  Writing 

height(<rfc(eA))  =  0 -r  height(<r,(eA, )), 
for  some  0  with  n  >  ^  >  0,  we  see  that 

ckmh + 0^^)) = ^'  = 

However,  because  -i-  0,  A)  occurs  to  the  left  of  5  in  the  query  tableau  representation 
of  Ci,  we  know  that  ^(.4*)  can’t  be  distinguished,  by  our  choice  of  9— a  contradiction. 

To  verify  condition  2  for  a  cut,  it  suffices  to  note  that  because  d*  is  a  persistent  variable 
which  arises  in  ^(9)  by  following  successor  s}rmbols  away  from  the  symbol  d  in  e^,,  the  vari¬ 
able  d  must  be  a  stem  or  persistent  variable  in  the  same  component  as  d*  of  the  substitution 
graph  of  Q. 

The  verification  of  condition  3  for  a  cut  is  immediate  by  our  construction,  because  we 
have  defined  d  as  equal  to  (r»(eA]),  where  ca,  has  an  A-leader  at  position  v. 

To  verify  condition  4  for  a  cut,  suppose  <r«(eA,)  =  do  is  persistent  or  stem.  Then  by  the 
definition  of  the  recursive  expansion  process,  9  has  a  persistent  variable  di  at  position  s,  and 
dg  and  di  must  belong  to  the  same  component  of  the  substitution  graph  of  Q.  Also,  because 
^  is  a  containment  mapping,  we  have  ^«(^(9))  =  di.  Since  di  can  appear  at  position  s  in 
^(9)  only  if  ex,  has  a  variable  from  the  same  component  as  di  at  position  s,  we  see  that 
fft{exi)  and  9«(eA,)  must  belong  to  the  same  component  of  the  substitution  graph  of  Q,  as 
required.  ^ 

To  verify  condition  5  for  a  cut,  suppose  cr«(eA, )  belongs  to  a  nondistinguished  variable  B , 
but  is  not  a  B-leader.  Now  suppose  that  contrary  to  the  statement  of  condition  5,  ^'((eA,) 
does  not  belong  to  some  nondistinguished  variable  C,  but  is  on  the  contrary  either  a  stem 
or  a  persistent  variable.  Then  ^*(^(9))  is  some  persistent  variable  do,  and  ^  carries  some 
to  do.  But  because  <r,(eA| )  does  not  have  maximal  height,  we  now  have  a  contradiction 
as  in  our  verification  of  condition  1  for  a  cut,  above,  by  our  choice  of  9. 
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FinJiy,  to  vorifj  condition  6  for  a  cnt,  1«  <-,(,»,)  b,  „  ^.leader.  Than  »,(,)  = 
».(«)  by  the  defmition  of  recarsiva  aipansion,  and  (r,(d(,))  at  o.(d(,))  =  f  becaiue  <t 

ts  a  containment  mapping.  Bacanaa  d  and  d  balong  to  tha  tama  componant  as  d  in  tha 
substitution  graph  of  g,  we  are  done.  | 


in  [a\aSa87] ,  Naughton  and  Sagiv  give  the  foUowing  four  conditions,  any  one  of  which  is 

individuaUy  sufficient  for  a  linear  sirup  Q  with  an  unbounded  head  chain  to  be  unbounded: 

A  For  no  subset  V  of  does  U,€V  =  V. 

B  C  contains  no  repeated  EDB  pr.  ilcates. 

C  No  persistent  or  stem  variable  appears  in  an  EDB  predicate  of  Q. 

D  For  no  argument  position  s  and  two  EDB  subgoals  e.  and  of  Q  does  have  a 
persistent  or  stem  variable  at  position  s,  while  e,  has  a  variable  belonging  to  some 
nondistinguished  A  at  position  s. 

We  claim  that  each  of  these  four  conditions  individually  implies  that  Q  has  no  cut;  the 
proofs  in  each  case  are  not  difficult: 

A  If  for  no  mb...  V  of  P.  do..  a.v  .(»)  =  V,  .hm  .h.  rpl.  S  h»  no  cycl..  in  it. 
.ub.t.tntion  gmph,  ud  henc.  hm  no  .ton  Old  no  per.i.toit  vaiiablo.  Since  th.  cut 
condition  requires  at  least  one  such  variable,  Q  has  no  cut. 

B  Strictly  speaking,  we  can  ignore  rules  with  no  repeated  EDB  predicates  because  it 
is  implicit  in  our  viewpoint  of  linear  sirups  that  they  involve  only  one  nonrecur- 
sive  predicate  name.  However,  it  is  usaful  to  see  how  the  Lemma  4.4  construction, 
even  when  presented  with  a  Datalog  rule  with  unique  EDB  predicate  names,  ensures 
that  the  corresponding  linear  sirup  Q  =  (ft.f)  can  have  no  cut.  First,  note  that  if 
^(C<(<i,Ai))  -  Ci(<j.A2))  is  part  of  a  containment  mapping  ^  for  such  a  rule,  then 
Ai  =  A,  is  forced.  If  we  suppose  then  that  such  a  linear  sirup  Q  had  a  (d,  A)-ait  ej  of 
some  other  subgoal  a,  then  we  see  immediately  by  condition  4  for  a  cut  that  e<  =  ej. 

Since  now  it  is  impossible  for  part  3  for  the  cut  condition  to  be  fulfilled,  we  have  a 
contradiction. 


C  If  no  persistent  or  stem  variable  appears  in  an  EDB  predicate  of  g.  then  it  is  impossible 
for  condition  2  for  a  cut  to  be  fulfilled 


<^UiI«i';t«*Sj>V^i4liiiiii>gitW-:^!tii*W»Ufcttflifai^^ 
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D  This  final  condition  is  the  one  which  most  closely  approximates  our  cut  condition.  In 
fact,  one  sees  that  the  stated  condition  is  a  slight  weakening  of  axioms  2,  3,  and  4  of 
the  cut  condition. 

It  should  be  observed  that  the  conditions  A,  C,  D  are  not  incommensurate:  in  fact 
condition  A  implies  condition  C,  while  condition  C  in  turn  implies  condition  D,  as  has 
been  observed  by  Naughton  and  Sagiv.  The  classes  described  by  conditions  B  and  C  are 
incommensurate. 

On  the  other  hand,  there  are  unbounded  rules  with  unbounded  head  chains  and  no  cut 
which  satisfy  none  of  the  [NaSa87]  conditions. 

Example  4.18  The  rule 

1A3  .422  /132  332  322, 

obtained  by  appending  the  distinguished  variable  2  to  each  EDB  subgoal  of  Naughton’s 
rule,  has  an  unbounded  head  chain,  satisfies  none  of  the  four  conditions  A*D  above,  but 
has  no  cut.  By  Theorem  4.15,  the  rule  is  unbounded.  I 


4.5  Cut  completion 

We  may  arrive  at  a  stronger  polynomial*time  testable  version  of  the  cut  condition  at  the 
expense  of  a  greater  time  complexity  in  testing  for  it  as  follows. 

Again,  let  2  be  a  linear  sirup  of  arity  n  with  k  EDB  subgoals,  and  suppose  that  2  has 
an  unbotmded  head  chain.  Let  e^  and  tj  be  two  EDB  subgoals  of  2,  and  suppose  that  tj 
it  a  (d,  A)-eat  of  e<.  We  say  that  ej  is  a  complete  (d,  A)-cut  of  e*  if  for  every  EDB  subgoal 
et  of  Q  that  contains  a  variable  belonging  to  A,  there  is  a  second  EDB  tubgoal  ey  (to  be 
called  a  (d,  A)*ciit  completion  of  e^)  that  satisfies  the  following  two  properties: 

1.  If  (r«(efc)  belongs  to  A  and  has  maximal  height  amongst  the  variables  of  ea  that  belong 
to  A,  then  (r«(efc>)  belongs  to  the  same  component  as  d  in  the  substitution  graph  of 

C- 

2.  If  belongs  to  A  and  does  not  have  maximal  height  amongst  the  variables  of 
that  belong  to  A,  then  <r,(efc>)  belongs  to  some  nondistinguished  variable  C. 
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Theorem  4.19  1/  Q  is  uniformly  bounded  and  has  an  unbounded  head  chain,  then  Q  has 
a  complete  cut. 

Proof  By  Theorem  4.15,  2  has  a  cut.  Suppose  then  that  the  EDB  subgoal  ex,  is  a 
(<f,.4)-cut  of  some  other  EDB  subgoal  e*,  of  Q.  It  will  be  our  assumption  that  the  e^j-eA, 
cut  arises  exactly  in  the  way  described  in  our  proof  to  Theorem  4.15 — that  is,  we  shall 
assume  that  there  are  recursive  expansions  and  Qj  of  2  with  t  <  j  and  a  containment 

mapping  d> :  Qt  -*  Qj  such  that  q  =  is  the  leftmost  EDB  subgoal  on  the  head 

chain  of  Qj  that  contains  some  nondistinguished  variable  at  an  argument  position  $  that 
maps  to  a  persistent  distinguished  variable  (f,  and  where  <>(g)  =  Qjiht  ^j)  fot  some  tj.  Also, 
again  as  in  Theorem  4.15,  we  may  additionally  assume  that  t  >  3n,  that  j  >  4nk{i  1), 

and  finally  that  ^(2»(<  -  =  QjU  ~  ^  ^  when  o  is  in  the  range  0  <  o  <  n. 

It  follows  therefore  that  ti  <  s'  -  n  here. 

In  the  proof  of  Theorem  4.15,  we  used  0,  q,  A',  and  d!  to  deduce  that  ca,  must  be  a 
(d,  .4)-cut  of  eA,  for  some  variable  d  in  the  same  component  as  of  the  substitution  graph 
of  2;  now,  it  will  be  our  concern  to  show  how  the  cut  completion  axioms  1  and  2  can 
additionally  be  deduced  for  the  present  set-up. 

Suppose  that  ex  is  an  EDB  subgoal  of  2  that  contains  at  least  one  variable  belonging 
to  A,  and  suppose  that  sq  is  chosen  so  that  9««(eA)  is  a  variable  of  maximal  height  amongst 
all  variables  of  ca  that  belong  to  A.  We  must  show  that  ca  has  cut  completion  ca'  in  2-  If 
the  A-leaders  of  2  have  height  0,  then  we  can  write  - 

0  =  height(<y„(cA))  +  r 
for  some  t  in  the  range  0  <  t  <  n,  so  that 

tr„(Qi(tj  -  r.A))  w  A‘  =  tr.(2i(f»,  A,))  =  <r.(?) 
by  the  definition  of  recursive  expansion*.  Now  suppose  that 

^(Ci(«i-T.A))=2i(lj.A'). 

We  claim  that  ca'  is  a  (d,  A)  cut  completion  of  ex  in  2- 

*It  may  be  asked  how  we  know  that  (i  -  r  >  0.  The  answer  is  that  because  «  contains  an  A-Ieader  at 
pontion  s,  we  know  that  (t  is  as  large  as  height (i4-Ieader)  -r  t,  while  r  is  smaller  than  height(^-leader). 
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T3 

In  proof,  we  turn  first  to  the  first  cut  completion  axiom,  which  is  easily  vermed;  because 

and  we  know  that  ^(.4')  ==  d\  which  belongs  to  the  same  component  as  d  in  the  substitution 
graph  of  C,  we  see  that  tx*  must  also  have  a  distinguished  variable  from  the  same  component 
as  d  at  position  sq^  in  order  for  d'  to  arise  at  position  sq  in  Q.j[h,  A'). 

To  verify  the  second  cut  completion  axiom,  let  tr,i(eA)  be  a  variable  that  belongs  to  .4, 
yet  (Tsiitx)  does  not  have  maximal  height  amongst  all  the  variables  of  that  belong  to  .4. 
Then  we  can  wite 

height((T,„(eA))  =  tq  -  height(cr.j  (e>)) 

for  some  0  <  tq  <  n.  Now  suppose  that  contrary  to  the  second  cut  completion  axiom,  we 
find  that  o'tjex*)  does  not  belong  to  a  nondistinguished  \'ariable,  but  is  on  the  contrary  a 
persistent  or  stem  variable.  Because 

o‘.,(Ci(«i  -r,A))  =  .4'  =  -t-to,A)), 

we  see  that 

is  a  nondistinguished  variable  of  that  ^  maps  to  a  persistent  distinguished  variable.  But 
because  the  symbol  ^4''*'^  also  occurs  in  2i((i  -r  ro,  A),  to  the  left  of  q  in  the  query  tableau 
representation  of  we  have  a  contradiction,  by  our  choice  of  q. 

I 

In  .he  first  line  of  the  figure  below,  we  rewrite  Naughton’s  rule  yet  again.  Underneath 
each  EDB  subgoal  that  contaiiu  a  variable  belonging  to  A,  we  have  written  a  cut  com¬ 
pletion  ei^  corresponding  to  the  (3,.4)-cut  of  by  ej.  Therefore  Kaughton’s  rule  has  a 
complete  cut,  in  accordance  with  Theorem  4.19. 

Ii43  A2  AZ  33  32 

AZ  33  33 

We  conclude  this  chapter  with  the  study  of  a  more  complex  example. 

Example  4.20  Consider  the  linear  sirup  2 

2i4B42  2i42  1.43  411  4AZ  445  444 


/ 
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of  arity  n  =  5  with  t  =  6  EDB  subgoals.  The  substitution  graph  of  2  is 


1 


and  one  can  ■  ~sily  check  by  hand  (or  verify  using  Naughton’s  algorithm)  that  Q  has  an 
unbounded  head  chain.  The  variable  4  is  the  only  persistent  variable,  while  the  variables 
1,  2,  a,  and  A  belong  to  A,  and  3  and  B  belong  to  B.  The  variables  1  and  5  are  both 
.<4-leaders,  and  each  has  height  two;  3  is  the  only  S-leader,  and  it  has  height  one.  There  are 
no  stem  variables. 

Now,  we  look  to  see  what  cuts  we  can  find  in  Q.  First,  we  note  that  =  2.42  can  be  cut 
by  no  other  EDB  subgoal,  because  it  contains  no  leader.  Next,  we  turn  to  ej  —  1>13,  which 
contains  the  two  leaders  <ri(ej)  =  1,  and  0'3(ej)  =  3.  Thus,  it  is  possible  that  there  may 
be  either  a  (4,.4)-cut  of  t2  by  some  other  EDB  subgoal,  or  alternatively  a  (4,B)-cut.  We 
consider  the  latter  possibility  first.  First,  if  is  a  (4,  B)-cut  of  1.43,  then  by  Axiom  3  for  a 
cut,  above,  we  have  <r3{e*)  =  4.  The  only  EDB  predicate  with  a  4  in  position  3  is  ce  =  444, 
but  unfortunately,  ee  also  has  a  4  in  position  2,  in  which  position  ej  has  the  nonpersistent, 
nonleader  variable  A.  Thus  Axiom  5  for  a  cut  would  be  violated,  so  we  conclude  there  is 
no  (4,B)-cut  of  cj. 

Next,  we  return  to  the  possibility  that  there  is  a  (4, 4)-cut  of  ej  in  Q.  Here,  we  meet 
with  initial  success;  the  subgoal  =  4 A3  is  a  (4,A)*cut  of  Also,  we  find  that  the 
subgoals  €3  through  es  have  valid  completions: 

2ABA2  2A2  1A3  411  4  A3  445  444 

4A3  444  443  444 

but  then  we  find  ourselves  in  a  quandary  with  C}.  A  (4,  A)  cut  completion  tk  of  ei  must 
have  4  m  positions  1  and  3,  and  a  nonpersistent  variable  in  position  2.  Because  there  is  no 
such  EDB  subgoal,  we  find  that  the  (4,  A)*cut  of  ti  by  is  not  complete. 

So  far,  we  have  ruled  out  the  existence  of  complete  cuts  for  ti  and  for  ej.  Are  there 
complete  cuts  for  any  of  the  remaining  four  EDB  subgoals  e3,e4,e8,  or  ee?  We  can  exclude 
ee  immediately,  because  it  contains  no  leader.  The  subgoal  es,  however,  contains  two 
occurrences  of  the  A-leader  variable  1.  We  may  hope  therefore  for  a  (4,  A)-cut,  but  we 
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already  know  that  tx  has  no  (4,j4)-cut  completion,  to  there  is  no  point  in  continuing. 
Turning  next  to  e^,  we  find  that  it  contains  the  B-leader  3  at  position  3.  Is  there  a  (4,B)- 
cut  of  e4  in  C?  The  answer  is  no— the  argtiment  may  be  carried  out  just  as  we  did  when  we 
considered  a  (4,B)-cut  for  e,,  above.  FinaUy,  what  about  e*?  The  variable  5  at  position  3 
in  es  is  an  .4-leader,  and  ee  =  444  is  a  (4,i4)-cut  of  6$,  but  again,  because  such  a  cut  has 
no  completion,  we  find  our  path  is  blocked. 

In  short,  Q  contains  no  complete  cut  whatsoever.  Our  final  conclusion  (via  Theo- 
rem  4.19):  Q  is  unboxmded.  I 


Chapter  5 

Rule  factorization 


In  this  chapter,  we  shall  examine  the  problem  of  rule  factorization.  Just  as  in  our  previous 
chapters,  we  shall  be  restricting  our  attention  to  finite  nonempty  rule  sets  S  whose  members 
are  linear  recursive  Datalog  rules  over  a  single  recursive  predicate  p. 

5.1  Factorization  and  the  equation  r  =  rjrj 

We  recall  some  deiinitioiis.  We  define  linear  recursive  Datalog  rules  Vi  and  to  be  equivalent 
(and  write  rj  or  rj)  if  they  simultaneously  stand  in  the  relationships  ri  >  vj  and  rj  v  ri. 
Two  rules  ri  and  rj  are  said  to  be  isomorphic  (and  we  write  tj  =  rj)  if  ri  and  rj  are 
identical  up  to  a  renaming  of  the  their  nondistinguished  variables  [Naug86b].  If  r  is  the 
rule  obtained  by  syntactically  expanding  the  rule  ri  by  the  rule  rj,  then  r  and  r^rj  are 
isomorphic  and  we  shall  say  that  f  admits  the  factorization  r  =  rirj. 

In  its  most  general  form,  the  rule  factorization  problem  can  be  simply  stated:  “Given 
a  rule  r,  how  can  we  find  rj  and  vj  so  that  r  =  rirj?"  Because  most  of  our  theorem 
statements  from  Chapter  3  assumed  such  factorizations  to  be  given,  the  rule  factorization 
problem  it  basic  to  the  applicability  of  our  techniques. 

Intuitively,  we  may  tbinV  of  rule  factorization  as  “the  inverse  of  recursive  expansion,"  but 
there  are  some  important  technical  points  to  be  considered  if  we  are  to  think  of  factorization 
in  these  terms.  Most  importantly,  note  that  the  factorization  equation  r  s=  rjrj  is  to  be 
interpreted  in  .7(5),  where  iS  =  and  net  in  the  rule  expansion  semigroup  .7(5)  itself. 

Because  the  rules  r  and  rjrj  in  a  factorization  r  =  rjr,  are  identical  up  to  the  renaming  of 
nondistinguished  variables,  it  follows  easily  that  r  2:  riVj,  but  the  converse  is  not  true  in 
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general.  The  diffictilty  arises  because  equivalence  does  not  imply  isomorphism.  Studying  the 
factorization  equation  r  =  r  jrj  is  therefore  different  from  studying  the  algebraic  relationship 
r  i  rjr2.  The  distinction  arises  because  general  rules  may  involve  redundant  EDB  subgoals. 

Example  5.1  The  three  rules 

r  :  p(XiX2)  :  -  p(AB)  It  e(X2Xi)  t  e(XiX2)  k  e(X2Xi) 

>•1  :  p(XiX2)  :  -  p(X2Xi)  t 

Ti  :  p(X,X2)  :  -  p(AB)  t  e(X2Xi) 

stand  in  the  relationship  r  ir  rjr2,  but  r  is  not  isomorphic  to  rii*2  because  r  contains  a 
redundant  occurrence  of  the  EDB  subgoal  e(X2Xi),  while  rjr2  does  not.  Strictly  speaking, 
the  equivalence  r  2:  rir2  is  therefore  not  a  factorization  (although  it  is  certainly  an  algebraic 
relationship,  and  we  could  make  it  into  a  factorization  by  deleting  one  of  the  redundant 
EDB  subgoals  from  r).  I 

To  summarize,  we  might  say  “factorization  is  defined  relative  to  isomorphism,  and  not 
equivalence.”  There  are  both  ads’antages  and  disadvantages  with  taking  this  viewpoint. 
On  the  positive  side,  factorization  is  perhaps  more  naturally  thought  of  as  the  inverse 
of  recursive  expansion  than  it  is  in  terms  of  the  algebraic  velationship  r  s  rjr2,  and  we 
shall  be  able  to  find  efficient  factorization  algorithms.  On  the  other  hand,  an  unfortunate 
consequence  of  our  definition  is  that  whether  a  nontrivial  factorization  exists  for  an  element 
in  a  ^  equivalence  class  of  ,7(5)  may  depend  on  the  particular  rule  expansion  r  chosen  from 
that  class. 

Example  5.2  The  three  rules 

** :  perils)  :  -  p(X2Xi)  »  •(AC)  t  e(AB)  t  e(X2B) 

p(*iX2)  :  -  p(X2Xi)  t  •(X2k) 

*•2  :  p(XiX2)  :  >  p(XiX2)  t  •(XjB)  t  .(IjC) 

stand  in  the  relationship  r  2:  tiT2,  but  t  cannot  be  written  isomorphically  as  a  recursive 
expansion  r  =  r'r"  of  two  rules  r'  and  r"  with  at  least  one  EDB  subgoal  apiece.  (See  the 
following  two  sections  for  additional  explanation  and  proof  method).  However,  we  can  write 
r  2:  ro  =  rir2,  where  tq  is  the  rule 


>•0  :  p{XiX2)  :  -  p(X2Xi)  t  «(X2B)  k  «(X2C)  k  e(X2A). 
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Another  point  we  must  consider  before  we  attempt  to  develop  a  factorization  theory  is 
that  a  factorization  may  be  "degenerate”  in  the  sense  that  one  of  the  two  factors  rj  or  rj 
may  act  as  an  identity  element: 

Example  5.3  The  simple  transitive  closure  rule 

r  :  p(XiX2)  :  -  p(XiA)  t  e(AX2) 

admits  the  factorization  r  =  fit;,  where  the  two  rules  fj  and  r2  are  defined  as  follows: 

n  :  p(XiX2)  :  -  p(XiA)  t  •(AX2) 
rz  :  p(XiX2)  ;  -  p(XiX2) 

Here,  r2  is  essentiaUy  “the  identity,”  while  rj  is  just  r  itself.  A  good  factorization  theory 
should  identify  such  factorizations  as  trivial  in  some  sense.  I 

To  sidestep  these  two  difficulties,  we  shall  talce  the  definition  that  if  r  is  a  rule  with 
k  >2  EDB  subgoals  and  rj  and  T2  each  have  at  least  one  nonrecursive  subgoal  apiece,  then 
we  shall  call  a  factorization  r  =  rir2  nontrivial-,  otherwise,  the  factorization  is  said  to  be 
trivial.  The  definition  not  only  ensures  that  degenerate  factorizations  involving  rules  with 
no  EDB  predicates  will  be  identified,  but  also  that  every  rule,  even  one  with  redundant 
EDB  predicates,  wiU  have  a  factorization  into  irreducible  rules,  as  described  in  the  next 
section. 


5.2  Irreducible  rules 

When  a  rule  r  admits  no  nontrivial  factorization  r  =  rir2,  we  caU  it  irreducible. 

Example  5.4  The  rule 

*• :  p(XiX2)  :  -  p(AX2)  *  eCAXi)  k  .(XiA) 

is  irreducible.  In  proof,  suppose  to  the  contrary  that  there  is  a  nontrivial  factorization 
r  =  rir2.  Then  rj  and  r2  each  have  the  form 


p(X,X2)  :  -  p(— )  t  •(— ), 
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and  we  must  decide  how  to  “fill  in  the  blanks”  so  that  r  and  rirj  are  isomorphic.  There 
are  two  possible  expansion  tree  forms  for  such  a  factorization.  The  first  possibility  is  that 
v/e  have  an  expansion  tree  of  the  form 


Po(A'i,A'2) 


e2(-lA'i,-!.4) 


where  the  e(lXi)  of  r  occurs  in  rj,  and  e(XiA)  arises  upon  expansion  of  by  r2-  (We  have 
written  the  expansion  tree  with  unsimplified  and  as  yet  partially  indeterminate  substitution 
chains,  and  we  have  put  subscripts  on  the  various  occurrences  of  the  predicate  names  e  and 
p  in  the  tree  only  in  order  to  distinguish  between  them  more  easily). 

The  other  possibility  is  that  we  have  an  expansion  tree  of  the  form 


Po{Xi,Xi) 


P3(-h.-|-)  ei[XuA) 


l-IJT,) 
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where  the  roles  of  e(iAi)  and  e(XiA)  are  reversed. 

To  see  why  neither  expansion  tree  form  above  can  correspond  to  an  actual  recursive 
expansion  of  rules,  we  focus  on  the  subgoal  substitutions  of  r^.  Inspecting  either  pj  and 
62  from  the  first  expansion  tre<;,  or  alternatively  P4  and  from  the  second,  we  see  in  each 
case  that  rj  must  have  three  subgoal  substitutions  (-jA,  -jA'],  Because  the  three 

blanks  to  be  filled  in  in  the  last  expression  must  be  occupied  by  three  distinct  distinguished 
miables,  we  have  a  contradiction — the  arity  ofp  is  only  two,  so  ri  has  just  two  distinguished 
variables.  Therefore  r  is  irreducible. 

I 


We  can  generalize  the  notion  of  rule  factorization  in  the  natural  way  to  consider  factor¬ 
izations  into  more  than  two  rules.  By  a  simple  inductive  argument  we  can  show: 

Theorem  5.5  Every  linear  Datalog  rule 

r  ;  p  :  -  p  5  ei  k  •  •  •  t 

is  either  irreducible  or  is  expressible  as  a  factorization  r  =  rirj  •  •  -  r,  of  irreducible  rules  ri 
for  some  a  >  2. 

Proof  We  use  induction  on  k,  the  number  of  EDB  subgoals  that  occur  in  r.  The 
theorem  is  clearly  true  if  ib  =  1.  Assume  it  is  true  for  every  integer  k  <  N.  Then  if  r  is 
not  irreducible  we  may  write  r  =  r'r"  where  r'  and  r"  each  have  at  least  one,  and  fewer 
than  k,  EDB  subgoals  apiece.  Hence  the  inductive  hypothesis  applies,  and  both  r'  and  r" 
are  either  irreducible  or  are  themselves  expressible  as  a  product  of  irreducibles.  Therefore 
r  too  is  a  product  of  irreducibles,  as  required.  I 

We  pause  again  to  look  at  another  example. 

Example  5.6  The  "same  generation  query”  {RSUVb9] 

f  :  p(XiX2)  :  -  p(AB)  t  •(XjA)  k  •(XaB) 

can  be  factored  into  irreducibles  as  r  =  rirj,  where 


ri  :  p(XiX2)  :  -  p(XiA)  k  e(X2A) 
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and 


rj  :  p(XiX2)  :  -  p(AX2)  t  e(XiA) 

are  commuting  rules  that  we  have  met  before.  In  fact,  t*i  is  precisely  the  rule  d  from 
Example  3.4,  while  r2  appears  in  Examples  2.8  and  3.5.  I 

It  is  our  belief  that  most  “natural”  linear  recursions  will  factor  in  simple  ways;  of  course, 
there  is  no  ultimately  decisive  way  that  we  can  defend  our  claim.  But  the  last  example  query 
is  perhaps  one  point  in  our  favor.  Although  the  same  generation  query  is  often  put  forward 
as  a  “nontrivial”  recursion  (in  particular  because  it  is  “2-sided”  in  the  sense  of  Naughton), 
we  find  that  it  factors  naturally  as  the  product  of  two  1-sided  recursions  that  themselves 
play  roles  in  the  generation  of  classes  of  recursions  to  which  our  semigroup  techniques  have 
been  seen  to  apply.  Thus  it  is  perhaps  not  unreasonable  to  expect  that  more  complex 
rule  sets  will  also  typically  involve  recursions  that  are  built  from  simpler  rules  by  recursive 
expansion. 

It  should  be  said,  however,  that  the  particular  r  =  rir2  factorization  of  the  same  genera¬ 
tion  rule  in  Example  5.6  does  not  have  immediate  implications  for  its  efficient  evaluation.  In 
particular,  the  commutative  decomposition  does  not  improve  upon,  or  indeed  say  anything 
directly  about  the  “counting”  algorithm  [Ban86a]  or  other  algorithms  for  same  generation 
rule  evaluation.  However,  two  points  can  be  made.  First,  the  factorization  r  =  rir2  does  re¬ 
veal  some  information  about  the  structure  of  the  same  generation  rule  that  can  illuminate 
the  process  of  writing  similar  logical  rules  (see  Section  5.5).  Second,  every  factorization 
r  =  rir2  has  the  potential  to  reveal  redundancy  in  settings  where  r  appears  in  conjunc¬ 
tion  with  other  rules  (which  themselves,  we  hope,  may  also  admit  factorizations  into  parts 
involving  vj  and  ^2  so  that  our  methods  in  Chapter  3  will  apply). 

Next,  we  shall  take  up  the  factorization  problem  from  a  constructive  point  of  view. 

5.3  Bounded  arity  factorization 

Example  5.4  contains  the  germ  of  an  idea  for  an  algorithm  that  factors  a  bounded  arity 
linear  recursive  rule  in  time  polynomial  in  the  size  of  the  rule. 

Theorem  5.7  Suppose  r  is  a  linear  recursive  Datalog  rule  whose  recursive  arity  n  does 
not  exceed  a  fixed  positive  constant  K.  Then  r  can  be  either  recognized  (u  irreducible  or 
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constructively  factored  as  a  nontrivial  product  r  =  rjrj  in  time  where  N  is  the 

size  of  the  rule  in  standard  Datalog  notation. 

Proof  We  shall  assume  that  the  rule  r  takes  the  form 

r:p 

with  recursive  arity  n  <  K  and  k  EDB  subgoals.  We  may  view  the  problem  of  factoring  r 
as  one  of  building  an  abstract  expansion  tree  rjrj  of  the  form 


Po 


Pi  ««  •••  ej. 

w'here  >  1,  s  +  *  =  i,  and  r  and  rirs  are  identical  up  to  renaming  of  their  nondistin* 
guished  variables  and  the  rearrangement  of  their  subgoals.  Again,  we  have  put  subscripts 
on  the  three  occurrences  of  p  in  the  expansion  tree  only  in  order  to  distinguish  between 
them  more  easily. 

Our  method  will  take  the  following  form:  first,  we  shall  present  a  nondeterministic 
algorithm  for  the  factorization  problem,  and  then  we  shall  use  the  bounded  arity  assumption 
to  show  how  the  algorithm  may  be  made  to  be  deterministic  and  still  run  in  polynomial 
time. 

Suppose  for  the  moment  that  we  are  given  a  guessed  trial  correspondence  mapping  ij) 
between  the  subgoals  of  r  and  the  subgoals  of  the  expansion  tree  rif].  We  shall  require 
the  mapping  df  to  be  (in  part)  a  bijection  between  the  k  EDB  subgoals  {ci.C], . .  ..Ck}  of  r 
and  the  k  EDB  subgoals  , . . . ,  Cj, , . . . ,  ej,  of  the  expansion  tree  rirj  such  that  and 
^(e,-)  are  identical  up  to  a  fixed  renaming  of  nondistinguished  \'ariables  that  is  independent 
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of  the  choice  of  i.  We  must  require  the  rcciirsive  subgoal  correspondence  v(p)  =  pj  between 
r  and  rjrj  to  respect  the  fixed  nondistinguished  ^-ariable  renaming  as  well.  Intuitively, 
the  mapping  v  simply  specifies  for  each  e,  whether  it*  syntactic  counterpart  in  is  to 
be  found  at  the  first  (corresponding  to  rj)  level  of  the  tree,  or  alternatively  the  second 
(corresponding  to  recursive  expansion  of  rj  by  rj). 

Having  guessed  the  correspondence  v,  we  wish  to  know:  “can  the  partial  expansion  tree 
corresponding  to  xif  be  completed  to  conform  to  an  actual  expansion  of  two  rules 
We  saw  in  Example  5.4  how  a  simple  necessary  condition  for  the  completion  of  the  tree 
may  be  given  by  counting  subgoal  substitutions  of  ri.  More  precisely,  for  every  distinct 
distinguished  \-ariable  Xa  that  occurs  either  in  pj  or  in  some  we  saw  that  ri  must 
have  a  subgoal  substitution  of  the  form  We  begin  therefore  by  counting  the 

number  c'  of  distinguished  variables  that  occur  cither  in  p^  or  in  some  Next,  if  /I  is 
a  nondistinguished  variable  that  occurs  either  in  p2  or  in  some  Cj^,  and  .4  also  occtirs  in 
some  ,  then  we  know  that  pi  must  contain  an  occurrence  of  .4,  and  that  has  a  subgoal 
substitution  of  the  form  (-1^4)^^.  Let  the  number  of  such  nondistinguished  variables  .4  be 
c”.  The  condition  from  Example  5.4  was  then  exactly  this:  the  sum  c  =  can  be 

no  greater  than  the  recursive  arity  n  of  the  rule  r,  for  otherwise  the  expansion  tree  cannot 
exist. 

In  fact,  we  claim  that  the  stated  condition  is  also  sufficient  for  the  completion  of  a  trial 
correspondence  as  well.  In  other  words,  we  claim:  a  trial  correspondence  ^  will  correspond 
to  an  actual  recursive  expansion  rirj  if  and  only  if  we  have  c'  -r  c"  <  n. 

The  claim  is  easily  proved  by  direct  construction  of  the  rules  rj  and  f2.  Suppose  that 
the  c  =  c^  +  variables  accounted  for  in  the  two  counts  arc  c'  distinguished  variables 
A'aii •  •  M and  c"  nondistinguished  variables  j4i,...,i4e»».  We  shaU  call  the  c'  -  c" 
variables 

{ ,  • .  • ,  , .  •  • ,  J 

the  charged  variables  of  the  trial  correspondence  if. 

We  now  turn  to  the  construction  of  Because  c'  +  c"  <  n,  we  may  construct  the 
first  c'  +  c"  positions  in  the  recursive  subgoal  of  ti  by  placing  all  the  charged  variables 

ia  the  first  e*  +  c*'  variable  positions.  Any  remaining  variable 
positions  of  the  recursive  subgoal  of  rj  are  filled  in  identically  by  putting  the  variable  Xi  at 
every  position  i,  where  c'  +  c"  <  t  <  n.  The  nonrecursive  subgoals  of  are  made  to  match 
their  trial  correspondence  counterparts  in  r  exactly  (recall  that  no  subgoal  substitutions  are 
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applied  to  these  subgoals  when  rj  is  expanded  by  r2).  So  far,  we  have  forced  the  expansion 
tree  to  agree  with  r  at  its  top  level. 

Now,  w’hat  about  r^?  We  turn  first  to  the  construction  of  its  nonrecursive  subgoals. 
At  positions  where  charged  variables  are  to  occur  in  rir2,  we  must  be  certain  that  the 
appropriate  substitution  ^akes  place;  thus  the  correct  must  be 

placed  at  every  such  position  in  the  EDB  subgoals  of  r2.  Any  remaining  variable  positions 
in  the  nonrecursive  subgoals  of  r2  are  necessarily  occupied  by  nondistinguished  variables 
that  appear  in  no  ti,.  If  W  is  such  a  variable,  we  can  replace  all  occurrences  of  \V  in 
P2  and  the  nonrecursive  subgoals  of  f2  by  ^be  symbol  B*  for  some  implicit  substitution 
of  ri  and  new  nondistinguished  variable  symbol  B.  (A  nondistinguished  t-ariable 
renaming  W  —  B*  has  taken  place  here).  If  the  sjmbol  W  (now  renamed  B')  occurs 
somewhere  in  p2i  say  at  position  t,  then  {Xi  B)^^  becomes  a  substitution  of  r2,  and  the 
unsimplified  substitution  chain  at  position  /  in  p2  bas  the  form  {Xi\B\B^)^^^^.  The  remaining 
substitutions  of  r2  are  determined  by  the  substitutions  of  ri  in  a  similar  manner — if  the 
partial  substitution  chain  at  position  I  in  p2  is  then  {Xi[Xt^)^^  becomes 

a  substitution  of  r2.  There  can  be  no  possible  conflict  between  such  definitions  because 
each  r2  subgoal  substitution  is  applied  only  once  in  the  expansion  tree  (in  the  appropriate 

position  of  P2  ) 

W’e  have  proved  our  claim,  above,  but  the  accomplishment  is  a  modest  one.  All  we 
know  is  that  if  someone  suggests  to  us  how  to  split  the  EDB  subgoals  of  r  between  ri  and 
r2  by  giving  us  a  trial  correspondence  then  we  can  determine  w’hethcr  the  suggestion 
corresponds  to  an  actual  recursive  expansion,  and  we  can  construct  a  corresponding  expan¬ 
sion  tree,  if  one  exists.  But  because  there  are  essentially  on  the  order  of  2*  possible  trial 
correspondences  to  be  considered,  we  do  not  yet  have  an  efficient  algorithm  for  the  bounded 
arity  factorization  problem. 

We  shall  overcome  the  difficulty  by  guessing  not  the  trial  correspondence  itself,  but 
instead  merely  the  identitieM  of  its  charged  variables. 

Recall  that  the  linking  graph  <J  of  r  is  an  undirected  graph  on  the  subgoal  vertices 
p,ei, . . . ,Cfc,  with  an  edge  between  two  subgoals  if  they  share  a  nondistinguished  variable. 
Here,  we  shall  introduce  a  colored  linking  muliigtaph  T  of  r  whose  edges  are  again  determined 
by  shared  nondistinguished  variables,  but  where  these  edges  are  also  colored  according  to  the 
particular  nondistinguished  variable  shared.  Thus  between  two  vertices  of  the  multigraph 
we  may  have  several  edges,  one  of  each  of  several  colors,  each  color  corresponding  to  some 
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applied  to  these  subgoals  when  rj  is  expanded  by  rj).  So  far,  we  have  forced  the  expansion 
tree  to  agree  w’ith  r  at  its  top  level. 

Now,  what  about  VVe  turn  first  to  the  construction  of  its  nonrecursive  subgoals. 
At  positions  where  charged  wiables  arc  to  occur  in  rir2,  we  must  be  certain  that  the 
appropriate  substitution  iA*,  -Yo./Vj  takes  place;  thus  the  correct  A',  must  be 

placed  at  every  such  position  in  the  EDB  subgoals  of  Any  remaining  wiable  positions 
in  the  nonrecursive  subgoals  of  rj  are  necessarily  occupied  by  nondistinguished  variables 
that  appear  in  no  c»,.  If  W  is  such  a  variable,  we  can  replace  all  occurrences  of  W  in 
P2  and  the  nonrecursive  subgoals  of  by  the  symbol  B*  for  some  implicit  substitution 
of  fi  and  nett?  nondistinguished  variable  symbol  B.  (A  nondistinguished  variable 
renaming  W  B*  has  taken  place  here).  If  the  symbol  \V  (now  renamed  5')  occurs 
somewhere  in  pj,  say  at  position  1,  then  {Xi  B)^^  becomes  a  substitution  of  r2,  and  the 
unsimplified  substitution  at  position  /  in  p2  has  the  form  The  remaining 

substitutions  of  r2  are  deiermined  by  the  substitutions  of  rx  in  a  similar  manner— if  the 
partial  substitution  chain  at  position  I  vx  p2  is  then  becomes 

a  substitution  of  r2.  There  can  be  no  possible  conflict  between  such  definitions  because 
each  r2  subgoal  substitution  is  applied  only  once  h\  the  expansion  tree  (in  the  appropriate 
position  of  P2  ) 

We  have  proved  our  claim,  above,  but  the  accomplishment  is  a  modest  one.  All  we 
know  is  that  if  someone  suggests  to  us  how  to  split  the  EDB  subgoals  of  r  between  rx  and 
r2  by  giving  us  a  trial  correspondence  then  we  can  determine  whether  the  suggestion 
corresponds  to  an  actual  recursive  expansion,  and  we  can  construct  a  corresponding  expan¬ 
sion  tree,  if  one  exists.  But  because  there  a^e  ess^^  :tially  on  the  order  of  2*  possible  trial 
correspondences  to  be  u;  idered,  we  do  no*  yet  have  an  efficient  algorithm  for  the  bounded 
arity  factorization  problem. 

We  shall  overcome  the  difficulty  by  guessing  not  the  trial  correspondence  itself,  but 
instead  merely  the  identitieM  of  its  charged  variables. 

Recall  that  the  linking  graph  G  of  r  is  an  undirected  graph  on  the  subgoal  vertices 
p,  ex, . . . ,  Cfc,  with  an  edge  between  two  subgoals  if  they  share  a  nondistinguished  variable. 
Here,  we  shall  introduce  a  colored  linking  muliigraph  F  of  r  whose  edges  are  again  determined 
by  shared  nondistinguisht  J  variables,  but  where  these  edges  are  also  colored  according  to  the 
particular  nondistinguished  variable  shared.  Thus  between  two  vertices  of  the  multigraph 
we  may  hV'e  several  edges,  one  of  each  of  several  colors,  each  color  corresponding  to  some 
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different  nondistinguished  variables  the  two  subgoals  share. 

Let  C  =  {Aa, . A'o,,.  .4, . .4,,,}  be  a  guessed  set  of  charged  s-ariables  corresponding 

to  some  trial  correspondence  between  the  EDB  subgoals  of  r  and  those  of  r;rj.  We  know 
that  it  can  be  assumed  that  c'  -  c"  <  n  here.  To  verifs-  the  guess,  we  must  chick  that  the 
EDB  subgoals  of  r  can  be  split  so  as  to  yield  exactly  the  charged  variable  set  C.  (Such 
a  spUt  15  in  fact  exactly  a  trial  correspondence  i  ).  We  may  apply  the  following  five  step 
T^multtgraph  algorithm. 


1.  Check  whether  all  distinguished  variables  of  p  are  in  C.  If  not,  answer  “no:  guess 
invabd,^  and  exit.  Otherwise: 


2.  Delete  aU  edges  colored  .4i ,  .4j, . . . ,  .4,..  from  T. 

3.  Call  the  resulting  graph  T'. 

4.  Compute  the  connected  components  T', .  Tj, . . .  TJ  of  T'  \  p. 

5.  US  >  land  r  .p  has  a  component  containing  no  uncharged  distinguished  s-ariables. 
then  answer  “yes:  verified  guess:”  otherwise,  answer  “no:  guess  invalid.” 


To  see  that  our  verification  algorithm  is  correct,  we  need  to  show  that  it  answers  “yes” 

if  and  only  if  the  EDB  subgoals  of  r  can  be  split  so  as  to  yield  exactly  the  charged  variable 
set  C. 

Suppose  first  that  the  EDB  subgoals  of  r  can  be  split  into  two  nonempty  sets,  one  of  “top 

level  EDB  subgoals  {cj, , . . . ,  e^,},  and  one  of  “second  level"  EDB  subgoals  {cj, _ _ e^, }, 

with  >  1,  a  *  «  =  and  charged  variable  set  C.  Then  by  the  definition  of  C  and  T', 
the  set  {cj, ,  cj, }  of  second  level  subgoals  contains  no  uncharged  distinguished  variables, 
and  there  can  be  no  multigraph  edge  of  T'  between  an  and  an  Ci,.  Similarly,  p  can 
neither  contain  an  uncharged  distinguished  variable  nor  share  an  edge  with  an  e*. ,  and  so 
p  must  belong  to  T'  also.  Therefore  P  and  {e>, , . . . ,  e^. }  together  induce  a  union  of  connect 
components  in  F,  these  vertices  do  not  include  all  of  T'  because  there  is  at  least  one  vertex 
e„  detached  from  these  vertices,  and  the  component  containing  can  contain  no  uncharged 
distinguished  variables.  Therefore  our  verification  algorithm  will  answer  “yes,"  as  required. 

Conversely,  suppose  that  the  algorithm  answers  “yes”  and  we  need  to  verify  that  there 
is  a  split  of  the  EDB  subgoals  of  r  with  charged  variable  set  C.  Here,  it  will  suffice  to 
identify  the  set  {cj, .....Cj,}  with  a  particular  connected  component  T',  of  T' '  p.  The 
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assumption  that  T'  \p  has  at  least  f  >  2  components  ensures  that  there  is  at  least  one 
remaining  EDB  vertex  in  the  components  r^.r^, . .  ..TJ.  Therefore  the  subgoals  of  any 
component  that  contains  no  imcharged  distinguished  variables  can  be  identified  with  the 
the  set  {c;, ,  —  Cj,}.  However,  the  charged  variable  set  C  corresponding  to  this  trial 
correspondence  is  only  necessarily  contained  in  C  and  need  not  be  C  itself.  Thus  strictly 
speaking  we  have  not  verified  our  guess  precisely,  but  because  IC'|  <  IC\  <  n,  the  charged 
variable  counting  condition  is  satisfied  for  C.  So  r  is  still  factorizable,  and  we  can  answer 
“yes”  for  this  guess  and  still  have  a  correct  algorithm. 

Because  there  are  no  more  than  charged  ^'ariable  set  selections  to  be  considered 
and  the  verification  algorithm  above  can  be  made  to  run  in  time  0(1V*),  our  entire  non- 
deterministic  factorization  algorithm  made  be  made  to  be  deterministic  and  run  in  time 
(5(;^z(A+i))  by  considering  aU  possible  charged  variable  sets  in  turn.  I 

There  is  no  assertion  in  Theorem  5.7  that  the  two  rules  rj  and  rj  are  themselves  ir¬ 
reducible.  If  a  factorization  into  irreducible  rules  is  desired,  one  can  apply  the  theorem 
iteratively  to  obtain  a  factorization  into  irreducibles  in  time 

Example  5.8  For  an  example  of  how  the  F-multigraph  algorithm  works,  consider  the  prob¬ 
lem  of  factoring  the  rule  , 

r  :  p(XiX2)  ;  -  p(XiD)  k  •(DDA)  k  «(AX2B)  k  f(BC). 

The  colored  linking  multigraph  of  r  is 


eiAXiB) 
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If  we  make  the  (erroneous)  guess  of  the  charged  variable  set  C  -  {Ai,P},  then  the 
graph  r*  becomes 


p(X,D) 


f{BC) 


•  e(DDA) 


and  r'  \p  is  seen  to  have  just  6  =  1  component.  So  the  algorithm  would  answer  “no:  guess 
invalid"  for  this  guess.  However,  r  does  have  a  factorization  corresponding  to  the  guessed 
charged  variable  set  {A'l.A}.  Here  the  graph  T'  becomes 

p(A'iD) 


f{BC) 


e{AXiB) 


we  have  ^  =  2  >  1  components  in  T'  \  p,  and  the  isolated  vertex  e{DDA)  in  T'  \  p  is  a 
component  containing  no  uncharged  distinguished  variables.  Therefore  the  T-multigraph 
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algorithm  would  answer  “yes:  verified  guess”  for  the  guess  C  =  {A'i,.4}.  Translating  the 
guess  into  a  factorization  as  suggested  in  the  proof  to  Theorem  5.7,  we  obtain  the  niles 

n  :  P(XiX2)  :  -  p(XiA)  t  t  ©(AXjB)  k  f{BC) 
r2  :  p(XiX2)  :  -  p(X,A)  t  eCAAXj) 

and  one  can  check  that  r  and  rirj  are  isomorphic.  I 


5.4  Unbounded  arity  factorization 

If  we  do  not  know  an  explicit  bound  on  the  recursive  arity  of  a  linear  rule  r,  then  the 
T-multigraph  factorization  algorithm  is  not  guaranteed  to  run  in  time  polynomial  in  the 
size  of  r.  The  difficulty  arises  because  the  ntimber  of  test  (guessed)  charg'd  variable  sets  to 
be  considered  can  no  longer  be  guaranteed  to  be  polynomiaUy  bounded  in  the  input  size. 

Nevertheless,  general  (unbounded  arity)  linear  recursive  rules  r  may  be  either  recognized 
as  irreducible  or  constructively  factored  as  a  recursive  expansion  r  =  rjrj  in  polynomial 
time  in  the  size  of  r  by  a  more  complex  method,  which  we  shall  call  the  flow  graph  algorithm. 

To  describe  the  method  we  shall  return  to  a  rule  notation  similar  to  that  used  in  Chapter 
4,  where  we  represented  distinguished  variables  by  positive  integers  and  suppressed  all 
predicate  names.  Because  the  factorizability  of  r  is  not  influenced  by  the  particular  predicate 
names  on  subgoals  but  rather  depends  only  on  how  variables  are  shared  between  subgoals, 
we  can  cast  the  irreducibility  problem  as  an  abstract  decision  problem  in  the  following  way. 

RULE  IRREDUCIBILITY 
INSTANCE: 

A  collection  €  of  |f|  >  3  finite  subsets  of  A/*  (J  A,  where  Af  =  {1,2,3,...}  and 
A  =  {A,B,C,...y,  also,  a  distinguished  element  p  £  €  and  a  coat  bound  c,  which  is 
an  arbitrary  positive  integer. 

QUESTION: 

Can  the  collection  f  \  p  be  partitioned  in  2  nonempty  parts  U  and  P  with  cost  less 
than  The  cost  of  a  l(,V  partition  is  computed  as  the  sum  of  all  variable  charges, 
a  .allows: 
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(DISTLN'GUISHED  VARIABLE  CHARGES) 

We  charge  1  for  every  n  £  A*  that  occurs  either  in  p  or  in  some  element  of  D. 

(NOXDISTINGUISHED  VARIABLE  CHARGES) 

We  charge  1  for  every  letter  q  €  that  occurs  both  in  some  element  oft/,  and 
in  either  p  or  in  some  element  of  P. 

We  shall  solve  the  rule  irreducibility  problem  by  reducing  it  to  a  certain  flow  problem 
in  graphs.  The  method  is  most  easily  envisioned  as  a  two-step  process.  In  the  first  step,  we 
shall  reduce  the  irreducibility  question  to  the  following  problem  in  bipartite  graphs. 

BIPARTITE  DISCONNECT  SET 
INSTANCE: 

A  bipartite  graph  G  =  (V,  E)  with  vertex  set  V  =  X'l  U  Ij,  and  1 1  2  »  empty.  All 

edges  e  €  £  have  one  endpoint  in  Vj  and  one  in  IV  Also,  two  distinguished  nodes  s 
and  t  in  I'l,  and  a  cost  bound  c,  an  arbitrary  positive  integer. 

QUESTION: 

Can  s  be  disconnected  from  t  in  G  by  removing  no  more  than  c  vertices  from  I  j  only? 

We  shall  first  show  how  an  instance  of  rule  irreducibility  of  size  n  can  be  polynomially 
reduced  to  the  problem  of  solving  0(n*)  instances  of  bipartite  disconnect  set.  Then  we  shall 
show  how  each  such  bipartite  disconnect  set  problem  can  be  solved  in  polynomial  time  by 
flow  techniques.  Cur  final  algorithm  will  have  complexity  0[v7),  where  again,  n  is  the  size 
of  the  input  rule  r  in  standard  Datalog  notation. 

Let  £  be  an  instance  of  the  rule  irreducibility  problem  with  distinguished  element  p  £  £ 
and  cost  bound  c.  We  seek  a  partition  of  £  \  p  into  nonempty  parts  U  and  V  that  has  cost 
at  most  e. 

Suppose  for  the  moment  that  we  guesi  from  the  set  £  \  p  the  identities  of  one  element 
tt  e  W  and  one  element  d£V.  It  is  our  claim  that  the  problem  of  deciding  whether  these 
initial  guesses  may  be  filled  out  into  a  complete  partition  of  £  \  p  meeting  the  given  cost 
bound  can  be  thought  of  as  an  instance  of  bipartite  disconnect  set. 

Here  are  the  details.  We  build  an  instance  of  bipartite  disconnect  set  as  follows.  First, 
we  create  a  node  vi  €  Vj  for  every  element  e  €  £,  and  we  create  a  node  vj  €  1*2  for  every 
svmbol  n  €  Af  or  a  €  that  occurs  in  some  element  of  £.  Next,  edges  are  drawn  between 
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vertices  vi  €  and  I'j  €  ^  j  if  and  only  if  the  symbol  vj  occtu^s  in  the  set  vj .  So  far,  we  have 
a  bipartite  graph  G  on  the  disjoint  vertex  sets  \'i  (J  1;  =  V*.  Each  edge  has  one  endpoint 
in  I’l,  and  one  in  IV 

Next,  two  new  vertices  s  and  t  are  added  to  TV  vertex  s  is  made  to  be  adjacent 
to  each  vertex  of  T  j  that  either  represents  a  distinguished  variable  n  €  A',  or  represents  a 
nondistinguished  variable  occurring  in  u.  The  vertex  t,  on  the  other  hand,  is  made  to  be 
adjacent  to  each  vertex  of  T'j  that  either  occurs  in  p  or  d. 

Our  claim  is  this:  the  given  starter  guesses  u  ^  U  and  d  €  P  can  be  filled  out  into  a 
complete  partition  of  5  \  p  that  meets  the  given  cost  bound  e  if  and  only  if  in  the  graph  G, 
the  vertices  s  and  t  can  be  disconnected  by  the  removal  of  at  most  c  vertices  from  T  j. 

We  shall  prove  our  claim  below,  but  first  it  is  useftil  to  look  at  another  example. 

Example  5.9  Consider  the  problem  of  deciding  whether  the  rule  r  given  by 

r  :  p(XjX2)  :  -  p(XiA)  *  •(A)  t  fCXjB)  t  f(CX2)  t  g(X2AB) 

is  irreducible.  We  first  convert  the  problem  into  the  simplified  notation  of  the  rule  ir* 
reducibility  problem  statement,  above.  Replacing  the  distinguished  variables  by  integers 
and  suppressing  all  predicate  names  in  every  subgoal  of  r,  we  obtain  the  collection  of  sets 
^  =  {ei,«2,e3,e4.«s}  given  by 


ei 

=  {l.A} 

=  {A} 

<3 

=  {2.B} 

u 

=  {2,C} 

€3 

=  {2,A.B}. 

The  rule  r  will  be  factorizable  if  and  only  if  the  set  f  \  p  s  {estesiesies}  can  be 
partitioned  into  two  nonempty  parts  U  and  V  with  a  cost  at  most  c  =  2,  the  recursive  arity 
of  the  rule  r.  In  this  particular  example,  there  is  such  a  partition — we  may  take  for  example 
U  =  {64},  and  T>  =  {62.63,64},  with  the  cost  charge  1{1,2}I  =  2  meeting  the  desired  bound. 
However,  note  that  some  care  must  taken  in  choosing  the  desired  partition — ^for  example, 
the  partition  IP  =  (es),  and  V  =  {63,63,64}  would  ful  to  meet  the  desired  cost  botmd, 
because  it  has  cost  charge  i{l,2.  A,  S}|  =  4  >  2. 
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r, 


1  = 


2  = 


>1  = 


B  = 


C  = 


Figure  5.1:  Bipartite  disconnect  set  problem 
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Suppose  we  correctly  guess  two  initial  members  of  the  former  partition  u  =  €4  €  W. 
and  <f  =  es  €  P.  It  was  our  claim  above  that  the  question  of  whether  this  initial  guess 
is  extendable  into  a  complete  partition  of  S  \  p  that  meets  the  desired  cost  bound  can  be 
thought  of  as  an  instance  of  the  bipartite  disconnect  set  problem.  To  illustrate  the  claim  in 
the  current  example,  in  Figure  5.1  we  have  drawn  the  associated  bipartite  graph  G,  whose 
desired  cost  bound  is  also  c  =  2.  Because  the  nodes  «  and  t  in  1 1  can  be  disconnected  by 
removing  the  two  nodes  1  and  2  &om  V],  the  bipartite  disconnect  problem  also  has  a  “yes” 
answer.  In  fact,  we  can  recover  the  partition  U,  V  by  taking  P  to  be  all  nodes  of  f  \  p  that 
are  connected  to  1  after  the  vertices  1  and  2  have  been  removed  from  G,  while  U  is  taken 
to  be  all  the  remaining  nodes  in  f  \  p. 

I 


Theorem  5.10  Ltt  5  =  {p  =  bt  an  instance  of  the  trreducibihty  problem 

with  cost  bound  c,  and  suppose  that  two  distinct  elements  u  =  Ci  and  <f  =  Cj  are  designated 
in  f  \p  =  {ej, . . . ,  Cfc}.  Let  G  —  G{S)  =  (I'l  'j  1  j,  £)  be  the  corresponding  constructed  graph 
instance  of  Vie  bipartite  disconnect  set  problem. 

Then  the  following  are  equivalent: 

(1).  The  vertices  s  and  t  can  be  disconnected  in  G  by  the  removal  of  e  vertices  from  I'j. 

(2.)  There  is  a  partition  of  £\p  into  two  nonempty,  disjoint  parts  U  (containing  u)  and 
P  ( containing  d)  whose  total  variable  charge  is  no  more  than  c. 

Proof  Show  that  (1)  ^  (2). 

Suppose  that  the  vertices  s  and  t  can  be  disconnected  by  the  removal  of  the  vertex  set 
C  C  Vi  from  G,  and  |C|  =  c.  Let  G'  be  the  resulting  subgraph  of  G.  Then  we  shall  construct 
a  Z/,P  partition  of  £  \  p  meeting  the  desired  conditions  as  follows.  We  let 

V  w  d  U  {v|v€f\pis  connected  to  t  in  (7'}, 
and 

=  u  U  {Everything  else  in  f  \  p}. 

We  must  verify  that  the  sets  U  and  P  are  indeed  a  disjoint  partition  of  t  \p  whose  total 
variable  charge  is  at  most  e. 
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First,  because  u  £  U  a: id  d  €  P,  we  see  that  U  and  V  are  nonempry,  and  by  construction 
L  and  V  together  exhaust  £  \  p.  Next,  we  must  show  that  U  and  V  are  disjoint.  By 
construction  of  the  graph  C,  if  t  is  any  node  in  G\  then  the  following  implications  are 
valid. 

1.  Vertex  x  is  reachable  from  d  in  G'  x  is  reachable  from  i  in 

2.  Vertex  x  is  reachable  from  u  in  G'  =>  x  is  reachable  from  j  in  G'. 

Therefore  U  V  must  be  empty,  for  if  x  is  in  1/  F  P,  then  x  is  connected  to  t  an  J  to  a, 
contradicting  the  fact  that  s  and  t  are  disconnected  in  G^ 

To  finish,  we  shall  show  that  the  charged  \“ariables  of  this  W,P  partition  are  all  neces¬ 
sarily  elements  of  C.  We  shall  consider  charged  distinguished,  and  then  nondistinguished, 
\*ariables  in  turn. 

Let  n  be  a  distinguished  \'ariable  charged  to  the  W,P  partition.  Then  n  cither  occurs 
in  p  or  in  some  element  of  P.  If  n  is  not  a  removed  vertex,  (i.e.,  an  element  of  C),  then  n  is 
a  vertex  of  Vj  in  G'  that  serves  to  connect  $  (which,  recall,  has  all  dl  distinguished  \-ariable 
adjacencies)  either  to  f,  or  to  a  vertex  connected  to  t.  Thus  we  would  conclude  that  s  is 
connected  to  t  in  G',  a  contradiction. 

Finally,  suppose  that  X  is  a  nondistingtaished  variable  charged  to  the  W,P  partition. 
Then  A  appears  both  in  some  v!  €  W,  and  also  in  some  d'  €  P.  Thus  if  the  variable  A  is 
not  one  of  the  deleted  vertices  C,  we  conclude  that  U  and  P  are  connected,  vrhich  is  again 
a  contradiction  by  the  construction  of  U  and  P. 

Show  that  (2)  (1). 

Let  W,P  be  a  partition  of  f  \  p  such  that  ti  G  W  and  d  €  P.  Let  the  charged  variable 
set  corresponding  to  this  partition  be  C.  We  claim  that  removing  the  vertices  C  from  Vj  in 
G  must  discozmect  $  from  t  in  the  resulting  graph  G\ 

In  proof,  suppose  to  the  contrary  that  there  is  simple  path  from  f  to  a  in  G'.  Vertices 
of  V)  that  appear  along  this  path  are  necessarily  uncharged  variables.  In  fact,  we  claim 
slightly  more  is  true:  .4ny  vertrx  of  \\  on  a  path  from  t  to  s  in  m%st  be  an  uncharged 
nondistinguished  variable.  Why?  First,  it  is  clear  that  no  vertex  at  distance  one  from  ( in  G' 
can  be  a  distinguished  variable,  because  such  a  variable  must  appear  either  in  p  or  in  d,  so 
therefore  is  charged  and  must  have  been  removed  from  G^  If  some  vertex  in  1 3  at  a  distance 
greater  than  1  from  t  along  the  path  were  a  distinguished  variable,  then  we  cotild  choose 
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the  closest  such  variable,  say  €  Tj,  at  distance  ^  >  2  from  t.  By  assumption,  no  vertex 
of  1 2  at  distance  <  6  from  t  is  a  distinguished  v^iablc,  and  we  know  each  is  uncharged. 
Therefore  each  verte:;  rj  \\  before  r2  on  the  path  from  f  to  s  must  correspond  either  to  p, 
or  to  an  clement  of  P.  Now  again  we  can  conclude  that  can’t  be  distinguished,  because 
otherwise  Vi  contains  an  imchaiged  disting'iished  variable,  and  is  either  p  or  an  element  of 
P. 

\Vc  have  showm  that  all  vertices  of  Ij  along  a  path  from  t  to  a  in  C  are  nondistinguished 
variables,  and  each  is  uncharged.  Therefore  any  \\  vertex  on  such  a  path  is  forced  to  be 
either  p,  or  an  element  of  P.  Since  in  particular  this  holds  for  the  last  vertex  (i  e.,  s)  on  the 
path,  we  conclude  that  u  itself  must  be  an  element  of  P,  which  is  a  contradiction  because 
the  sets  P  and  U  are  disjoint. 

So  no  path  from  5  to  t  exists  in  as  claimed. 


Therefore,  to  solve  instance  of  the  rule  irreducibility  problem  of  sire  n,  we  may 
consider  all  possible  “starter  pairs”  u  =  Cj  €  W  and  d  =  Cj  ^  P  in  turn.  For  each  starter 
pair  we  solve  the  correspondirg  bipartite  disconnect  set  problem  to  determine  if  the  given 
starter  pair  can  be  completed  to  form  an  actual  partition  of  £  \  p  meeting  the  desired 
cost  bound.  There  are  only  0(n*)  possible  starter  pairs  to  be  considered,  and  each  such 
constructed  bipartite  disconnect  set  problem  has  0(n)  vertices  and  O(n^)  edges.  We  shall 
therefore  have  a  polynomial  time  algorithm  for  rule  irreducibility  if  the  bipartite  disconnect 
set  problem  can  be  solved  in  polynomial  time. 

"We  shall  transform  a  given  instance  G  =  (li  U  Vj,  £)  of  bipartite  disconnect  set  into 
a  flow  problem  in  directed  graphs.  An  edge-weighted  directed  graph  D  with  new  source 
node  ao  and  sink  node  to  is  obtained  &om  G  as  follows.  First,  we  replace  each  node  v  €  1*3 
in  G  by  a  directed  edge  between  new  nodes  and  of  D.  The  capacity  of  each 

such  created  edge  is  made  to  be  equal  to  one.  Then,  for  each  edge  («;,  v)  of  G  with  tr  €  Vi 
and  V  €  Vi,  we  replace  this  edge  by  two  directed  edges  and  Each  of  these 

edges  is  made  to  have  capacity  inflnity.  Finally,  a  new  sink  node  so  and  a  sotirce  node  to 
are  created  in  P,  and  two  edges  of  infinite  capacity  (so,  s)  and  (f,to)  are  added  to  P. 

In  Figure  5.2  we  illustrate  the  construction  of  the  graph  P  from  a  particular  instance 
G  of  bipartite  disconnect  set. 

Suppose  we  use  a  known  polynomial  time  algorithm,  for  example  Dinits’s  algorithm 
[DiniTO],  to  find  a  maximal  flow  f  from  sq  to  to  in  the  network  P.  Then  excepting  the 
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edges  (ioi'j)  3^d  {t.to)  in  the  flow  F  must  be  a  0-1  flow  (i.e.,  every  edge  must  have  a 
flow  of  cither  0  or  1  along  if).  It  is  our  cliim  that  the  cardinality  C  of  a  minimal  bipaititc 
disconnect  set  C  Z  Tj  in  (7  is  e^ua!  to  the  cardinality  of  a  minimum  cut  in  the  flow- 
net  work  D. 

To  prove  the  claim,  wr  shall  show  that  the  inequalities 


M  <\C\<F 

are  valid.  By  the  max-flow  min-cut  theorem,  we  also  know  that  M  =  F,  so  thac  wc  shall 
be  able  to  conclude  that  M  =  |C1,  as  claimed. 

First,  wc  need  to  show*  that  M  <  \C\.  In  proof,  suppose  that  s  can  be  disconnected  from 
t  in  G  by  the  removal  of  the  vertices  C  C  TV  Then  a  (M*,  cut  of  the  network  D  with 
capacity  iC!  can  be  constructed  from  C  in  the  following  way.  (Recall  that  a  cut  in  a  flow 
netw’ork  D  is  a  partition  (I'T,  IF)  of  its  vertices  such  that  the  source  node  so  is  in  M*,  and 
the  sink  node  to  is  in  W\  its  capacity  is  the  sum  of  the  capacities  of  all  arcs  that  cross  the 
cut  in  the  “forward  direction*' — see,  for  example  [PaSt82]).  First,  we  put  Sq  in  H’,  and  we 
put  to  in  Then,  for  each  node  r  €  C  in  C,  we  put  the  corresponding  node  into  IT, 
and  we  put  the  node  into  W,  All  remaining  nodes  of  D  arc  put  into  W.  The  result  is 
a  (H^,  H’)  cut  of  the  netw’ork  Dy  and  its  capacity  is  precisely  |Ci.  Therefore  a  mimmal  cut 
necessarily  has  its  capacity  upper  bounded  by  |Ci,  as  claimed. 

Next,  we  need  to  show  that  |C|  <  F.  Here,  it  suffices  to  show  how  a  maximal  flow  in 
the  network  D  yields  a  disconnect  set  C  C  1  j  in  the  corresponding  bipartite  disconnect  set 
problem.  The  construction  is  simple.  Given  a  maximal  flow  in  D,  we  look  at  each  edge  pair 
jind  see  if  it  has  0  or  1  flow  along  it.  If  the  flow  is  1,  we  remove  the  corresponding 
node  V  €  t  a  bom  G,  and  if  zero,  then  the  corresponding  node  is  not  removed  in  G.  The 
given  nodes  must  disconnect  s  from  t  in  G,  for  otherwise  the  given  D  flow  can  be  augmented, 
contradicting  its  maximality. 

Dinits’s  algorithm  is  known  to  run  in  time  0(|rilA|*)  on  networks  with \V\  nodes  and  lA: 
arcs.  The  entire  flow  graph  algorithm  therefore  has  complexity  0{n*  x  n  x  (n^)*)  =  G{n^), 
as  claimed. 

Dinits*s  algorithm  is  known  to  run  in  timeO(iVIjAi^)  on  networks  with \V\  nodes  and  |.4' 
arcs.  The  entire  flow  graph  algorithm  therefore  has  complexity  0[t?  x  n  x  (n*)^)  =  O(n^). 


Bipartite  disconnect  set  as  a  flow 


*iC*l*iL-^** 
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5.5  Unique  factorization 

When  does  a  rule  admit  a  unique  factorization  into  irreducible  rules?  Unfortunately,  the 
answer  comes  back  fairly  quickly — not  very  often.  There  are  at  least  two  major  difficulties: 

1.  Rule  (non)commutativity:  Because  general  rules  do  not  commute,  we  must  think 
of  the  two  general  rule  products  uv  and  vu  as  "essentially  different.”  Yet  when  u  and 
V  do  happen  to  satisfy  uv  =  vu,  then  ur  and  ru  represent  the  same  rule.  Any  general 
unique  factorization  theorem  would  have  to  handle  this  difficulty  in  some  way. 

2.  Freedom  of  choice:  When  at  least  one  factorization  r  =  r^rj  exists  for  a  rule  r,  we 
often  shall  have  several  choices  a\*ailable  to  us  betw'een  different  selections  of  charged 
variables  in  the  rectirsive  subgoal  of  r^.  The  different  selections  ar^d  placements  of 
these  \-ariables  will  then  lead  to  other  factorizations  r  =  sjsj  where  si  ^  ri  and 
52  ^  r2.  For  example,  applying  the  F-multigraph  algorithm  to  the  same  generation 
rule 

r  :  p(XiX2)  :  ^  p(AB)  k  e(XiA)  k  e(X2B), 
one  may  obtain  the  factorization  r  =  5i52  with  the  rules 

5i  :  p(XiX2)  :  «  p(AXi)  k  e(X2A) 

:  p(XiX2)  :  -  p(XiA)  &  e(X2l). 

as  weU  as  the  factorization  r  =  rir2  from  Example  5.6.  The  charged  variable  set 
corresponding  to  both  factorizations  is  {Jfi,  A},  and  the  essential  difference  between 
them  is  the  order  in  which  these  two  variables  occur  in  the  top-level  recursive  subgoals 
Ti  and  5i. 

Amongst  the  multiple  factorizations  of  a  rule  we  can  sometimes  discover  curious  ways 
to  write  simple  logic  programs.  For  an  example,  suppose  we  set  ourselves  the  task  of 
writing  a  logic  program  11  to  compute  a  “same  generation  or  one  older”  relation  p  from  a 
given  “parent”  EDB  relation  par(jr,y).  More  precisely,  we  shall  require  the  program  11  to 
compute  all  tuples  p{X,  Y)  where  either  X  and  Y  have  a  common  parent  ancestor  at  the 
same  parent  depth  d  >  0,  or  X  and  Y  have  a  common  ancestor  a  at  the  depth  d  1  from 
X  and  at  the  depth  d  from  Y. 
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A  first  r  at  writing  might  yield  the  foUowing  program 


n  :  sg(XiX,) 

’•2  :  sg{XjX2)  :  -  sg(A,B)  k  par(Xj,A)  t  par(X2,B). 

’•3  ••  p(Xi,X2)  sg(Xi,X2). 

■  P(Xi,X2)  sg(X2,i)  t  par(Xi,A). 

Here,  we  can  think  of  the  program  H  as  first  computing  aU  the  same  generktion  facts 
by  usmg  a  basis  rule  r,  and  the  standard  same  generation  rule  r,.  Next.  H  computes  the 

relation  p  from  sg  by  mcluding  all  the  sg  facts  (rule  r,).  and  the  “off  by  one  generation- 
facts  are  thrown  in  too  (rule  r^). 

However,  the  interesting  thing  is  that  the  same  generation  rule  admits  a  factorization 
sg  =  s  ,  where  s  is  the  one-half  same  generation  rule 

»  •  8g(Xi.X2)  :  “  sg(X2.A)  A  par(Xi,A). 

There  is  a  conspicuous  similarity  between  the  rule  body  of  s  and  that  of  the  rule  in 
n.  In  fact,  the  entire  program  H  may  be  replaced  by  the  program  H'  =  {r,s} 

’•••p(Xi.Xi) 

*  '  P(Xi,X2)  :  -  p(X2,A)  a  par(Xi,A), 

wU4  compute  th.  d«i«J  reUUo.  p  directly.  H«.,  .to  ..„d„d  g.„cr«i»a  fac 

W1  e  computed  by  the  even-power  recursive  expansions  of  s,  while  the  “off  by  one”  facts 
will  be  picked  up  by  the  odd  powers  of  s. 


5.6  Graph  semigroups 

But  the  last  example  is  probably  best  regarded  as  a  curiousity-in  general,  the  absence 
of  unique  factorization  properties  for  rule  sets  proves  to  be  more  a  hindrance  than  a  help. 
However,  there  is  a  natural  context  in  which  semigroups  and  unique  factorization  properties 
can  be  exploited  to  study  the  query  containment  properties  of  a  rule  set.  We  shall  take 
loannidis’s  [Ioan89]  paHial  commutativity  pnblem  as  our  starting  point. 

loannidis  [Ioan89]  has  drawn  attention  to  the  problem  of  finding  “ways  to  take  advantage 
of  partial  commutativity,  i.e.,  when  the  transitive  closure  of  a  product  of  operators  is  to  be 
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computed,  only  a  subset  of  which  are  mutually  commutative.”  To  restate  the  problem  in  our 
language,  we  may  generally  ask  what  can  be  said  about  the  query  containment  properties 
of  rule  sets  5  =  {rj, . . .  ,rfc}  for  which  we  may  know  several  algebraic  relationships  of  the 
form  TiTj  2:  VjTi,  Such  relations  may  be  known  to  us  either  by  using  the  techniques  outlined 
above,  or  from  the  results  of  other  authors  [Ioan89],  [RSUV89i. 

In  the  semigroup  world,  such  a  set  of  abstract  relations  is  said  to  be  a  presentation  of 
a  graph  semigroup  [KMR82],  and  useful  results  from  this  literature  may  be  carried  over  to 
the  study  of  the  partial  conunutati /ity  problem  for  rtile  sets. 

Let  S  -  {n,. .  be  a  rule  set,  and  let  w  be  an  arbitrary  recursive 

expansion  of  the  rules  in  5.  If  5  is  strongly  free,  then  we  know  that  w  determines  the  rules 
’‘m  > >  “  *  > uniquely — that  is,  w  2:  then  s  =  t  and  it  =  j/  for  1  <  /  <  s 

(Theorem  3.8).  Therefore  it  makes  sense  to  say  that  w  (and  indeed  every  recursive  expansion 
of  the  rules  in  S)  has  a  unique  factorization  over  5. 

How^ever,  now  suppose  to  the  contrary  that  S  is  not  strongly  free,  but  instead  satisfies 
some  commutative  algebraic  relationships  of  the  form  rirj  cr  rjri.  Then  we  can  no  longer 
speak  directly  about  xmique  factorizations — for  example,  w  2:  rir2r3  ^  r2rir3  if  rj  and  r2 
commute.  However,  we  can  prove  unique  factorization  result  of  a  slightly  different  kind. 
We  need  a  preliminary  definition. 

Again,  let  5  =  {ri, . . . , r^}  be  a  rule  set,  and  let 

W  CZ  WiW2'-Wl  (5.1) 

be  a  recursive  expansion  of  the  rules  in  S  where  each  ti;,*  is  either  a  rule  of  S  or  alternatively 
Wi  is  itself  a  recursive  expansion  of  rules  in  5.  If  for  each  pair  Wj  with  1  <  t,  J  <  /  we  find 
that  WiWj  2:  WjWiy  then  we  call  the  expression  5.1  a  factorizaiion  (of  w)  into  commuting 
parts. 

It  is  our  present  goal  to  prove: 

Theorem  5.11  Suppose  S  =  is  a  rule  set,  and  suppose  that  every  nontrivial 

algebraic  relationship  satisfied  by  the  rules  of  S  is  a  consequence  of  relations  of  the  form 
rirj  cz  VjVi.  Then  (i)  there  exists  a  unique  factorization  of  any  recursive  expansion  w  £  J{S) 
into  commuting  parts,  and  (ii)  we  may  decide  membership  for  the  set  of  all  elements  of  S{S) 
that  commute  with  a  particular  expansion  w  in  polynomial  time. 

Proof  Treating  the  r^’s  as  formal  symbols,  define  a  graph  F  on  5  by  taking  {ri,rj)  as 
an  edge  off  if  and  only  if  and  Vj  do  not  commute.  Let  G  be  the  rule  expansion  semigroup 
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J{S),  and  take  w  €  G  as  a  typical  fonnal  word  over  the  alphabet  5.  Then  we  shall  use 
C(u>)  to  denote  the  centrdizer 

C(tt)  =  {wo  !  a'ott’  -  a-’U'o} 

of  It',  and  we  shall  use  r(u’)  to  refer  to  the  full  subgraph 

r(tt’)  =  {{ri,rj)  6  r  1  fi.Tj  occur  in  ti;} 

generated  by  the  vertices  which  occur  in  w.  Let  the  connected  components  of  r(tt’)  be 

and  let  Wi,  the  ith  commutative  projection  of  w,  denote  the  product  of  the  vertices  obtained 
from  w  by  deleting  all  vertices  not  in  ri(tt').  Then  [KMR82]  tf,  is  well-defined,  u'.tfj  i  ttjtt)< 
for  all  i,j  and  ^ 

tl?  S  IL’i. 
isl 

Finally,  we  define  ir(tf)  =  {iwi, . . . ,  tc,}  as  the  set  of  all  commutative  projections  of  w.  (The 
present  notations  should  not  be  confused  with  those  used  in  Section  4,  above). 

An  element  u;  €  (?  is  called  a  c-prime  if  w  cannot  be  expressed  as  a  nontrivial  product 
w  2:  uv  vu  of  two  commuting  factors  u,  v  €  G.  A  c^factorization  of  tv  is  an  equation 
w  ^  yi  ...yk  where  for  each  i  and  j  we  have  yiyj  n  yjyi.  We  can  now  translate  the  central 
result  from  [KMR82]  into  the  present  context.  Because 

Every  element  tv  6  J{S)  has  a  unique  c-factorization  into  c-primes, 

we  have  a  proof  of  part  (t)  of  the  theorem  statement.  In  fact,  it  is  possible  to  give  a  simple 
characterization  of  the  c-primes  that  occur  in  the  c-factorization  of  the  element  tv.  Let 
tv  e  G.  Then  there  exists  an  element  root(tv)  €  G  such  that  (root(tv))’"  =  tv  for  some 
m  >  0,  and  for  any  x  €  G  such  that  x"  =  tv  for  some  n  >  0,  there  exists  o  >  0  such 
that  root(tv)*  =  x.  (For  a  proof,  again  consult  [KMR82]).  One  shows  then  that  c-primes 
which  occur  in  the  c-factorization  of  the  element  tv  are  roots  of  the  cormnutative  projections 
tv,-  6  *(tv). 

Finally,  we  come  to  the  much  simpler  part  (ii)  of  the  theorem  statement.  Here,  it  is 
perhaps  not  surprising  that  to  test  the  commutatitrity  of  two  elements  tv  and  v,  it  suffices 
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to  attempt  to  transform  u’r  into  vw  by  a  sequence  of  interchanges  of  commuting  vertices 
Ti  and  rj.  A  simple  greedy  quadratic  time  algorithm  suffices,  although  we  shall  present  a 
more  sophisticated  (linear  time)  algorithm  in  Section  6.2  below.  I 

Chapter  6  in  part  presents  a  case  study  of  how  Theorem  5.11  may  be  applied  to  the 
study  of  a  commonly  encountered  class  of  rule  sets,  the  linear  chain  rules.  Still,  it  is  useful 
to  pause  at  this  point  to  give  an  example  of  a  particular  rule  set  to  which  Theorem  5.11 
applies. 

Example  5.12  Consider  the  three  rules 

r  :  p(XiX2X3)  :  -  p(XiX2A)  t  e{AX3) 

3  :  p(XiX2X3)  :  -  p(XiX2A)  A  f(AX3) 

(  :  p(XiX2X3)  :  -  p(AX2X3)  »  g(AX:) 

Here,  we  find  that  the  rules  r  and  5  do  not  commute,  while  both  s  and  t,  and  r  and  t, 
do  commute.  The  graph  F  is  therefore 


and  it  has  two  connected  components.  Every  recursive  expansion  w  of  the  rules  {r,  s,  t}  will 
therefore  have  a  unique  factorization  into  at  most  two  commuting  parts  that  are  obtained 
from  w  by  commutative  projection  onto  the  components  of  F;  for  example,  the  expansion 
w  =  trrstr  can  be  factored  as  w  s:  WiW2,  where  tui  =  rrar  and  u»2  =  tt. 


Finally,  it  is  interesting  to  note  that  the  centralizer  C(w)  of  a  particular  element  w  in 
a  graph  semigroup  G  is  a  subsemigroup  of  G  that  is  again  a  graph  semigroup  (that  is,  it  is 
presented  by  relations  of  the  form  r,r,-  =  (KMR82].  Thus  it  appears  that  to  optimize 

such  programs  using  rule  commutativity  properties  unavoidably  involves  graph  semigroup 
analysis  at  some  level. 


Chapter  6 


A  case  study:  linear  chain  rules 


UUman  and  \’an  Gelder  [UlVaSo]  define  an  interesting  class  of  rule  sets  whose  query  con¬ 
tainment  properties  can  be  studied  by  using  a  combination  of  factorization  techniques  and 
graph  semigroup  analysis.  These  are  the  rule  sets  whose  elements  are  linear  chain  rules. 
We  need  a  preliminary  definition. 

An  elementary  chain  rule  is  a  rule  of  the  fcnn 

p(XiX2)  ri(Xi,Ai)  t  r2(Ai,A2)  t  ■  ■  k  rk(Ak-i,X2) 

where  all  predicates  are  binary  and  Xi,Ai,...,Ak-iiX2  are  all  distinct  variables  [XJlVa85]. 
In  a  linear  elementary  chain  rule,  exactly  one  of  the  predicate  names  ri  is  p. 

Example  6.1  The  rule 

p(XiX2)  qi(Xi,Ai)  A  p(Ai.A2)  i  q2(l2,X2) 
is  a  linear  elementary  chain  rule.  I 

UUman  and  Van  Gelder  focus  primarUy  on  elementary  chain  rules  with  nonlinear  re¬ 
cursion  aUowed,  and  they  show  how  the  recursive  expansions  of  such  rules  can  be  simply 
modeUed  by  context-free  grammars.  The  latter  observation  is  then  used  to  demonstrate 
that  the  computation  of  the  TninirmiTn  model  of  such  programs  is  efficiently  paraUelizable 
(i.e.,  in  ^^C)  when  a  certain  polynomial  fringe  property  is  satisfied. 

In  the  linear  case,  that  the  TniniTnum  model  computation  is  in  A/’C  is  immediate  because 
such  rules  are  “thinly  disguised  transitive  closure”  rules  |UlYa83]  for  which  minimum 
models  may  be  computed  by  path  doubling  techniques. 
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Suppose  5  is  a  role  set  whose  elements  are  all  elementary  chain  rules,  here  momentar¬ 
ily  with  nonlinear  recursion  allowed  (i.e.,  multiple  r^’s  may  be  p).  If  one  attempts  to  use 
the  context  free  grammar  approach  to  study  query  containments  amongst  recursive  expan¬ 
sions  of  the  rules  in  5,  then  discovering  nontrivial  containments  is  as  hard  as  deciding  the 
ambiguity  problem  for  context  free  grammars,  which  is  known  to  be  undecidable. 

However,  we  shall  see  that  /inear  elementary  chain  rules  have  particularly  simple  factor¬ 
ization  properties,  even  when  we  widen  our  perspective  to  generalized  (i.c.  nonelement  ary) 
forms  of  such  rules.  These  factorization  properties,  together  with  graph  semigroup  tech¬ 
niques,  will  allow  us  to  present  nontrivial  query  containment  algorithms  for  such  rules.  \Vc 
need  another  definition. 

Definition  6.2  A  linear  chain  rule  is  a  rule  of  the  form 

p  :  -  n  I:  12  ft  •  •  ‘  t  Tk 

where  the  arities  of  the  p  and  ri  predicates  may  be  arbitrary,  and  the  following  conditions 
arc  satisfied: 

•  (CHAIN  CONDITION)  For  I  <  i  <  j  <  k,  the  predicates  Ti  and  rj  share  a  nondis- 
iinguished  variable  t/ j  =  i  +  1,  and  otherwise  share  none, 

•  (L/R  BLOCK  CONDITION)  Each  distinguished  variable  occurs  exactly  once  in  the 
body,  either  in  ti,  or  alternatively  in  Moreover  and  r^  each  have  at  least  one 
distinguished  variable  occurring  in  them, 

•  (LINEAR  CONFORMITY)  Exactly  one  ri  is  p.  Moreover,  i/  ri  =  p  Aos  tAc  same 
variable  occurring  at  positions  a  and  0,  then  the  distinguished  variables  Xa  and  X0 
must  either  both  occur  in  Ti,  or  alternatively  both  occur  in  Xk- 

Example  6.3  Of  the  many  linear  recursive  rules  we  have  considered  in  this  thesis,  only  a 
few  have  not  been  linear  chain  rules.  Here  are  some  of  the  linear  chain  rules  we  have  seen 
so  far: 

The  transitive  closure  rule  for  directed  graphs: 


p(XY)  :  -  p{Xl)  I  e(AY). 
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The  same  generation  rule: 

p(X,X2)  ;  -  «(XiA)  t  p(AB)  A  e(X2B) 

The  one-half  same  generation  rule: 

sg(Xi,X2)  8g(X2,A)  *  par(Xi,A). 

Left-right  transitive  closure  rules: 

p(XlX2)  :  -  P(XlC)  t  «(CB)  t  •(AB)  t  •(AX2) 

p(XiX2)  :  -  p(XiB)  A  e(BA)  A  •(X2A) 

Many  more  rules  meet  the  definition.  For  example: 

p(XiX2X3)  :  -  p(XjCX3)  A  «(CBBD)  A  l(DBAB)  A  g(AX2)  (6.1) 

p(:iX2X3X4)  :  -  8(XiA)  A  f(ABCD)  A  p(DEEE)  A  1(X4X2EX3)  (6.2) 

For  some  examples  of  rules  that  do  not  quite  meet  the  definition,  we  can  take  the  three 
rules 


P(X3X2X3)  :  -  p(XiCX3)  A  •(CBB)  A  l(BAB)  A  1(ACX2) 

p(XiX2X3X4)  :  >  •(XiA)  A  f(AX2CD)  A  p(DEEE)  A  X(X4X3EX3) 

p(XiX2X3X4)  :  -  «(XiA)  A  l(ABCD)  A  p(DEED)  A  f(X4X2EX3) 

which  violate  the  chain  condition,  the  block  condition,  and  the  linear  conformity  condition, 
respectively,  from  top  to  bottom.  We  have  underlined  the  violation  in  each  of  the  three 
clauses.  I 


Here,  our  interest  will  be  restricted  to  linear  chain  rules  that  satisfy  a  stronger  version 
of  the  chain  condition: 

•  (STRONG  CHAIN  CONDITION)  For  1  <  t  <  j  <  fc,  the  predicates  Xj  and  Xj  share 
a  unique  nondistinguished  variable  if  and  only  t/  j  s  i  -f- 1. 
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We  impose  the  strong  chain  condition  rather  than  the  normal  chain  condition  only  in 
order  to  ensure  that  the  linear  chain  rules  we  consider  wiU  have  convenient  factorization 
properties.  For  the  remainder  of  this  chapter,  u’e  shall  assume  all  linear  chain  rules  to  satisfy 
the  strong  chain  condition  (but  we  remind  the  reader  that  our  class  is  more  restrictive  than 
the  original  definition  XlVaSo]  specifies).  However,  note  that  the  classes  of  rules  mentioned 
above  do  meet  the  strong  chain  condition  (although  rules  6.1  and  6.2  do  not). 

Suppose  we  are  given  a  set  of  linear  chain  rules  5  =  {^i,*  ■  -  re}  defining  a  relation  p, 
and  we  arc  considering  two  finite-depth  recursive  expansions  •  •  •  and  -  of  the  rules 
S.  How  can  we  efficiently  test  whether  raf^  •  •  •  >:  r,rt  -1  Our  goal  in  the  present  chapter 
to  give  two  algorithms  for  such  query  containment  problems.  Our  methods  will  not  be 
guaranteed  to  capture  all  the  query  containments  such  a  rule  set  S  may  satisfy,  however. 
Still,  much  nontrivial  information  can  be  deduced. 

Of  the  two  algorithms  we  present,  the  first  will  be  the  simpler,  and  will  depend  only  on 
the  unique  factorization  properties  present  in  graph  semigroups.  However,  the  first  method 
will  has  the  disadvantage  of  not  exploiting  all  the  commutativity  information  that  is  present 
for  general  linear  chadn  rules. 

The  second  algorithm  wUl  use  a  more  ad\-anced  technique  due  to  P.  Cartier  and  D. 
Foata  [CaFo69],  and  in  a  strong  sense  can  be  said  to  exploit  “everything  we  know  about 
commutativity”  for  linear  chain  rules  in  the  study  of  query  containments. 

The  following  3  obser\'ations  will  be  important  to  our  methods. 

1.  Linear  chain  rules  have  a  particularly  simple  factorization  structure.  In  fact,  every 
linear  chain  rule  with  ib  EDB  subgoals  can  always  be  efficiently  and  completely  factored 
into  a  product  of  k  irreducible  rules  with  1  EDB  sub  goal  apiece. 

2.  There  is  a  strong  commutativity  test  for  rules  with  1  EDB  subgoal  ;RSU\  89]. 

3.  In  light  of  points  1  and  2,  the  graph  semigroup  method  from  Chapter  5  may  be  brought 
to  bear  on  the  study  of  query  containments. 

6.1  Query  containment  by  commutative  decomposition 

Our  first  method  for  query  containment  testing  is  most  easily  illustrated  with  an  extended 
example. 
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Example  6.4  We  shall  take  the  rule  set  5  =  r,}  consisting  of  the  four  linear 

chain  rules 

r.s  :  p(XiX2X3X4)  :  -  PlXiAXsB)  t  e(CA)  ft  i{Cl2U) 

r,  :  p(XiX2X3X4)  :  -  pCHjXaB)  ft  1(X,AX4) 

Td  :  p(XjX2X3X4)  :  -  p(XiX2AX4)  ft  •(AX3) 

r.  ;  p(XiX2X3X4)  :  -  p(XiX2AX4)  ft  «(X3A) 

as  moderately  complex  running  example.  First,  one  quickly  verifies  that  the  strong  chain, 
block,  and  linear  conformity  conditions  are  satisfied  for  each  of  these  three  rules.  The 

notation  we  have  chosen  the  first  rule,  rgs,  stems  from  the  fact  that  it  has  a  complete 

factorization  into  single  EDB  subgoal  rules  =  r^rj,  where  the  latter  two  niles  are  given 
by 

r.  :  p(XiX2X3X4)  :  -  p(X,AX3B)  ft  f(AX2X4) 

rj  :  p(XiX2X3X4)  :  -  p(XiAX3X4)  ft  •(XjA). 

We  shall  see  below  that  such  complete  factorizations  always  exist  for  linear  chain  rules,  and 
that  they  may  be  found  efficiently.  The  mutual  commutativity  relationships  i  for 
the  live  niles  {ro,rk,re,rj,r,}  can  be  summarized  by  a  graph  T 


a 


b  e 


where  (in  a  manner  similar  to  Theorem  5.11)  we  have  drawn  edges  between  vertices  that 
do  not  commute.  Agw,  we  shall  see  below  that  this  commutativity  information  may  be 
efficiently  computed  in  the  general  case  by  applying  a  criterion  due  to  Ramakrishnan,  Sagiv, 
UUman,  and  Vardi  (RSUV89].  The  graph  F  has  connected  components  Fi  =  {a,6,c}  and 
F2  *  {d.e}. 

Now,  suppose  for  the  sake  of  an  example  that  we  would  like  to  test  whether  the  query 
containment  toi  >  holds  between  the  recursive  expansions  wi  =  tvz  = 
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TjrohTtTf  of  the  rules  b'.  Still  following  the  graph  sciTiigroup  method  from  Theorem  5.11, 
we  begin  by  calculating  the  commutative  projections 


abdee  dabce 


abc  de  abc  dt 


of  the  words  wiaud  iv}  onto  the  components  F]  and  Fj.  Because  we  find  that  tvi  and  tvj  have 
identical  factorizations  into  the  commuting  factors  r  =  and  »  =  r^r,,  we  conclude 

that  the  query  containment  does  indeed  hold  amongst  the  recursive  expansions  of 

S. 

To  take  a  second  example,  suppose  we  are  interested  in  testing  the  query  containment 
t»3  ^  «'4i  where  W3  and  Wt  are  the  recirrsive  expansions  W3  =  rerjesr,  and  =  r*rerfcrj. 
Here  the  respective  commutative  projections  are 


edbe  eebd 


eh  de  eh  ed 


and  because  the  projections  onto  Fa  diiTer,  (we  have  de  ^  ed),  we  cannot  conclude  that  the 
given  query  containment  holds  (and  in  fact  it  does  not). 

I 

To  complete  our  description  of  chain  rule  query  containment  testing  by  commutative 
decomposition,  we  need  to  check  the  the  points  left  incomplete  above.  First,  we  claimed 
above  that 
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Lemma  6.5  Every  linear  chain  rule  r  with  k  EDB  subgoals  can  be  factored  completely  as 
a  recursive  expansion  of  k  rules,  each  of  which  has  J  EDB  subgoal  apiece. 

Proof  Suppose  the  rxile  r  has  recursive  arity  n  and  takes  the  form 

r  :  p  :  ri  i  ra  t  *  ^ 

where  we  shall  suppose  that  it  is  the  subgoal  rj  in  the  body  of  r  that  corresponds  to  the 
occurrence  of  the  recursive  p  subgoal.  By  renaming  nondistinguished  variables  if  necessary, 
we  may  also  assume  that  the  nondistinguished  variable  shared  by  and  is  named  .4^ 
for  each  value  i  with  1  <  i  <  it. 

We  can  prove  the  lemma  directly  by  presenting  a  k  level  abstract  expansion  tree  for  r, 
where  at  each  level  exactly  one  EDB  subgoal  cT  f  appears.  The  structure  of  the  tree  is  as 
suggested  by  Figure  6.1. 

Here,  the  \-ariable  patterns  in  the  leaf  nodes  . .  .,r*  correspond  exactly  to  their 

counterparts  in  the  rule  r,  and  all  we  have  to  do  is  demonstrate  how  variables  may  be 
p^sed  do^Ti  through  the  intermediate p-subgoals pi,p2, .  •  fPj^ifPkfPk^it*  •  as  to 

guaraintee  that  the  resulting  tree  does  indeed  correspond  to  an  actual  recursive  expansion 
of  rules. 

We  consider  the  intermediate  p-subgoals  in  turn,  starting  with  pi-  Clearly  we  must 
take  Pi’s  \*ariables  as  A'i,X2, .  ••tXn  in  order,  because  this  is  forced  on  us  at  the  root  node 
of  all  expansion  trees.  Next,  for  indices  i  in  the  range  2  <  t  <  j  —  1,  we  must  first  take 
Pi  to  contain  the  nondistinguished  variable  Ai  that  is  to  be  shared  between  Vi  and 
Also,  every  distinguished  variable  that  is  to  occur  in  must  also  appear  somewhere  in  p*. 
Because  by  definition  can  contain  at  most  n  — ’  1  different  distinguished  variables,  there 
are  at  most  n  variables  (nondistinguished  and  distinguished)  all  told  that  must  appear  in 
Pt,  and  there  is  room  for  all  these  **ariables  amongst  the  n  argument  positions  of  pt. 

Next,  we  turn  to  pk»  This  subgoal  must  contain  Ak^\  together  with  whatever  distin- 
guished  variables  are  to  appear  in  Again  there  are  at  roost  n  such  variables,  so  they  all 
fit  inside  pk* 

Finally,  for  the  indices  i  in  the  range  *  -  1  >  i  >  J,  the  subgoal  Pi  must  contain  the 
two  variables  A,  and  Aj-i,  the  nondistinguished  variable  that  is  to  be  shared  between 
and  Tj.  No  distinguished  variables  need  appear,  because  all  such  variables  have  already 
occurred  in  ti  and  above  pi  in  the  expansion  tree. 
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Figure  6.1:  An  expansion  tree  for  a  chain  rule  with  tj  =  p. 
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CHAPTER  6.  A  CASE  STUDY:  LINEAR  CHAIN  RULES 

Secondly,  there  is  also  the  matter  of  testing  for  commutativity.  In  'RSUV89],  the  fol¬ 
lowing  simple  condition  is  presented. 

Theorem  6.6  Consider  two  safe,  Datalog  linear  rules  of  the  form 

ri  ;  p(Xi,...,Xn)  p(Yi,. .  .,Yn)  ft  Gj  t  •  •  •  t  G„ 

rj  ;  p(Xi,...,Xn)  p(2j, . .  ..Za)  ft  Hj  ft  ■  •  -  ft  Ht 

Then  the  following  conditions  are  sufficient  for  the  commutativity  law  rir2  =  r2ri; 

I-  A'i  then  Xi  does  not  appear  among  the  H 's. 

2.  If  Zi  ^  Xi  then  Xi  does  not  appear  among  the  G ’s. 

3.  IfYi  =  Xj  for  some  j  jt  i,  then  Zi  =  A',-  and  Zj  =  Xj. 

4-  If  Zi  =  Xj  for  some  j  ^  i,  then  yj  s=  Xi  and  Yj  =  Xj. 

The  theorem  gives  us  a  simple  method  to  test  for  commutativity  between  the  constituent 
factors  of  a  set  of  linear  chain  rules.  It  should  be  observed,  however,  that  because  we  are 
always  testing  for  commutativity  between  two  single  EDB  rules,  there  is  no  real  loss  of 
efficiency  in  the  alternative  of  simply  expanding  the  two  rules  rj  and  pj  in  both  orders  and 
testing  all  possible  query  containment  mappings  between  them  exhaustively. 

6.2  The  F-decomposition 

The  commutative  decomposition  method  fails  to  recognize  some  simple  query  containments 
that  are  consequences  of  partial  commutativity.  For  example,  in  Example  6.4  the  decom¬ 
position  method  would  fail  to  discover  that  r,rt,rc  t  Even  though  this  containment 

follows  immediately  from  the  relationship  rsPe  2:  PcPs,  the  projections  onto  the  connected 
component  Fj  differ  (and  in  fact  are  abe  in  the  first  case,  and  aeb  in  the  second). 

In  order  to  capture  aU  th;  consequences  of  partial  commutativity  we  may  apply  a  second 
idea  called  the  V -decomposition,  first  described  by  Cartier  and  Foata  [CaFo69). 

Suppose  that  5  =  {r*,  rs, . . .}  is  a  finite  rule  set  whose  mutual  commutativity  relation¬ 
ships  TiTj  ss  TjTi  have  been  summarized  by  a  graph  F  on  the  vertices  {a,6,. where  as 
usual  we  put  edges  between  vertices  that  do  not  commute.  We  shall  say  that  a  query  con¬ 
tainment  if>i  >  tt>j  between  recursive  expansions  tuj  and  tuj  of  the  rules  S  is  a  consequence 
of  partial  commutativity  if  the  word  tt»i  can  be  transformed  into  the  word  wj  by  a  sequence 
of  interchanges  of  commuting  rules  r.-pj  2:  VjVi. 
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begin 

Create  a  single  cell  with  the  first  letter  of  w  in  it. 
for  i  =  2  to  length(w)  do  begin 

(1)  Let  V  be  the  ith  letter  in  w. 

(2)  Find  the  rightmost  cell  c  that  either  contains  the 
letter  v,  or  contains  a  symbol  u  not  commuting  with 
V.  Then  place  v  in  the  next  cell  to  the  right  of 
c,  respecting  the  alphabetical  order  within  this 
cell.  If  c  is  already  the  rightmost  cell,  then  create 
a  new  rightmost  cell  with  v  only  in  it.  If  c  does  not 
gjfist ,  then  place  v  in  the  leftmost  cell ,  again 
respecting  the  alphabetical  order  within  that  cell. 

end 

end 


Figure  6.2:  Computing  the  V-decomposition  T(tt-)  of  a  word  w. 

We  shall  need  to  put  an  arbitrary  (but  fixed)  strict  linear  order  on  the  alphabet  in 
order  to  describe  F-decompositions.  Here  we  shall  take  the  convenient  alphabetic  ordering 

a  <  b  <  c  <  •  •  •• 

The  V -decomposition  of  a  word  w  over  the  alphabet  fl  =  {a,6, . . .}  is  a  symbol  \  (u-)  of 
the  form 

tt»l  1  I  .  .  .  I  Wk, 

where  the  Wi's  are  also  words  over  the  alphabet  SI.  The  u»<’s  are  called  the  celU  of 
To  compute  F(ti>)  from  u»,  we  apply  the  algorithm  in  Figure  6.2. 

Example  6.7  We  shall  compute  the  F-decomposition  V{w)  of  the  word  w  s  ahedaeccead, 
where  mutual  commutativity  relationships  are  summarized  by  the  graph 
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Following  the  algorithm,  we  obtain  the  following  sequence  of  cell  expressions  through 
each  pass  of  the  main  loop: 

a 

alb 
a  I  be 
ad  I  be 
ad  I  be  I  a 
ad  I  6ce  I  a 
acf  I  6ce  I  a  i  c 
ad  i  bee  I  a  I  c  I  c 
ad  I  bee  I  ae  I  c  I  c 
ad  j  bee  I  ae  I  e  I  e  I  a 
ad  I  bee  I  ae  I  ed  I  c  I  a 

The  final  expression  gives  the  T'-decomposition  of  the  word  it;.  We  have 

V(w)  ^  ad  I  bee  I  ae  I  cd  I  c  I  a. 


^-decompositions  are  relevant  to  the  study  of  query  containments  because  of  the  fol¬ 
lowing  two  results  [CaFo69]. 

1.  Ignoring  the  cell  dividers  in  we  have  a  single  word  that  is  commutatively  equiv¬ 
alent  to  ta. 

2.  Two  words  wi  and  taa  are  commutatively  equivalent  if  and  only  if  their  V  decompo¬ 
sitions  coincide. 

Thus,  to  test  whether  a  given  query  containment  wi  >  W2  is  true  as  a  consequence 
of  partial  commutativity,  we  may  simply  compute  F(u;i)  and  and  see  if  they  arc 

identical. 

Thus,  in  a  strong  sense  the  V -decomposition  captures  **everything  that  we  know  about 
query  containments  from  partial  commutativity.” 
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We  can  sum  up  our  Chapter  6  results  in  the  following  way.  We  have  given  two  methods 
for  recognizing  query  containments  between  the  recursive  expansions  of  linear  chain  rules, 
one  based  on  commutative  projection,  and  the  other  based  on  the  T-decomposition.  In  the 
particular  case  w^here  all  the  algebraic  relationships  of  a  set  of  linear  chain  rules  are  conse¬ 
quences  of  partially  commutative  relationships  r^rj  2:  we  have  shown  how  any  desired 

algebraic  relationship  can  be  tested  efficiently,  answ^ering  a  question  asked  by  loannidis. 
However,  the  theory  is  moderately  complex  and  it  is  not  at  all  clear  how  an  all-purpose 
algorithm  for  linear  chain  rule  e\^uation  could  be  devised  to  exploit  all  the  redundancy 
that  can  be  discovered  using  our  techniques.  Certainly  ad  hoc  techniques  as  one  finds  for 
example  in  [I0W088]  can  be  derived  for  particular  rule  sets  once  some  redundancy  has 
been  identified,  but  whether  general  purpose  algorithms  capable  of  exploiting  all  the  known 
redundancy  for  general  rule  sets  must  remain  a  topic  for  future  research. 


Chapter  7 


Conclusion 


There  are  several  open  problems  and  possibilities  for  extension  of  otir  work  that  we  would 
now  like  to  consider  briefly. 

Can  a  workable  algebraic  theory  of  nonlinear  recursive  rules  be  devised?  The  work 
of  loannidis  and  Wong  represents  an  important  first  step  in  this  direction.  The  major 
difficiilties  appear  to  arise  with  the  nonassociatc  character  of  nonlinear  rule  composition, 
so  that  the  strongest  theorems  are  proved  when  certain  associativity  assumptions  are  made 
[I0W088].  Saraiya  [Sar89a]  discusses  conditions  for  the  replacement  of  nonlinear  rules  by 
linear  ones  via  ZYT  linearization.  If  a  nonlinear  rule  set  can  be  ZYT  linearized,  is  there 
an  “inherent  associativity”  in  the  original  nonlinear  rule  forms  that  awaits  an  algebraic 
characterization? 

Is  linear  rule  set  freeness  decidable?  As  we  remarked  above  in  Chapter  4,  the  problem 
is  closely  related  to  results  on  boundedness  [Cosm88],  (GMSV87],  [loanSS],  [NaSa87].  Work 
on  boundedness  can  seem  to  be  far  removed  from  real  life  recursions  at  times,  but  it  is  fim- 
damental  to  our  understanding  of  the  redimdance  and  expressibility  properties  of  recursive 
database  queries. 

In  a  sequence  of  three  papers,  A.  Richard  Helm  [Hel87],  [Hel88a],  [Helm88b]  has  de¬ 
veloped  a  theory  of  detecting  and  eliminating  redimdant  derivations  in  logic  programming 
systems.  He  uses  a  control  language  over  derivations  to  illustrate  how  under  certain  cir¬ 
cumstances  redundant  derivations  may  be  made  not  to  occur  when  a  program  is  executed 
using  his  synthesized  control  expressions.  Because  his  control  strategies  are  essentially  reg¬ 
ular  expressions,  it  seems  likely  that  redundancy  discovered  using  not  only  the  techniques 
developed  here,  but  also  those  of  other  authors  [Dong88],  1I0W088],  could  be  meshed  with 
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Helm’s  control  strategies.  We  have  not  studied  this  idea  in  the  present  work,  but  it  does 
seem  to  be  a  natural  direction  for  future  research. 


Appendix  A 

Containment  depth  details 


In  this  appendix  we  complete  the  details  to  our  proof  of  Theorem  4.13,  above,  checking  the 
six  points  of  the  containment  mapping  definition  in  turn. 

(1) .  Check  that  ^,(/»<)  =  V-*’. 

This  is  immediate  from  the  definition. 

(2) .  Check  that  A))  6  for  all  t  and  A. 

This  is  also  immediate  from  the  definitions. 

(3) .  Check  that  if  is  a  distinguished  variable,  then  <rj(0j(h*))  =  <ri{h^~^)  is  the  same 

distinguished  variable. 

Suppose  that  <ri{h')  is  a  distinguished  variable.  Because  ^  is  a  containment  mapping, 
=  <Ti{h^)  is  the  same  distinguished  variable.  Next,  note  that  j  >  Ki  =  4pnki 
implies  that 

3  -p>  Spnki  >  n, 

i.e.,  j  -  p  >  n.  We  therefore  may  apply  Lemma  4.7,  part  1,  above,  to  conclude  that 
=  (ri(h^)  =  <ri{h%  as  required. 

(4) .  Check  that  if  is  a  distinguished  variable,  then  Ai)))  is  the 

same  distinguished  variable. 

Let  ^((?i(li,Ai))  =  Qi(f2,A3).  Two  cases  arise,  according  to  whether  f j  >  M  or  ts  < 
M  —  pn  +  1. 

Suppose  first  that  tj  >  Af.  We  have  already  noted  above  that  M  >  n,  and  in  the 
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verification  of  (3),  above,  we  saw  that  j  -  p  >  n.  Lemma  4.8,  point  1,  therefore  applies, 
and  we  conclude  Aj))  =  £rj(Qj(t2,  A2))  =  <r/(Cj-p(t2  Ai))), 

as  required. 

On  the  other  hand,  suppose  <2  <  M  -  pn  +  1.  Then  we  can  add  p  to  (2  without  making 
h  >  j-  Applying  Lemma  4.6  a  total  ofp  times,  we  conclude  cr((2i(ti)Ai))  =  = 

«^/(2i-p(<2,A2)),  as  required. 

(5).  Check  that  if  Ai))  =  ffJ,(C<(<2.^2))  then  Aj)))  is  equal  to 

0’/j(^l(Ci(t2.-^2)))- 

Let  (j>{Qi{ti,Xi))  =  Cj(3i,7i))  and  let  (^(Ci(t2.  A2))  =  C>(a2.72)- 

If  <^i,(Qi(ti,Ai))  =  c'/,(Q»(t2>'^2))  is  a  distinguished  variable,  then  the  verification  is 
carried  out  as  in  (4),  above.  So  suppose  that  ^.’i(C»(ti,Ai))  and  0‘i,(Ct(t2.  A2))  are  both 
equal  to  the  nondistinguished  variable  Because  ^  is  a  containment  mapping,  we  know 
that  V  =  o’f,(0(2i(ti,Ai)))  is  equal  to  <T/,(<^(Ci(t2i  ^2))).  If  the  variable  v  is  distinguished, 
then  we  may  again  reduce  our  argument  to  that  made  in  (4),  above.  So  we  can  suppose 
without  loss  of  generality  that  v  is  some  nondistinguished  variable 

Next,  we  claim  that  either  both  Si,S2  >  Af  or  both  si,S2  <  •Af  -  pn  -i-  1  is  the  case; 
i.e.,  it  is  impossible  that  we  have,  say,  a\  >  M  while  S2  <  M  -  pn  -i-  1.  (Recall  that 
by  construction,  neither  sj  nor  32  falls  in  the  closed  interval  (ilf  -  pn  -j- 1,  A/]).  To  prove 
this,  we  apply  Lemma  4.11.  We  have  |A/  +  1  -  (Af  —  pn  +  1)  +  1|  =  pn  +  1  >  n,  so  that 
B™  €  ®'(Qi(si>7i))  <^<’’(2i(sj»72))  implies  that  either  3i,s2  >  Af  or  3i,S2  <  Af  -  pn  1. 

To  finish  the  argument,  then,  we  must  again  consider  two  subcases.  In  the  first  subcase 
we  shall  assume  si ,  «2  >  Af ,  and  in  the  second,  ,  32  <  Af  -  pn  + 1. 

Suppose  first  that  31,32  >  Af.  Because  j  -p>n  and  31,32  >  Af  >  n,  we  may  apply 
Lemma  4.8,  part  2  twice  to  conclude 

=  jm-p 

=  <^l,(Ci-p(S2-P,72)) 

=  <^«a(^l(2i(t2.A2))), 

as  required. 

Finally,  suppose  that  31,32  <  Af  -  pn  +  1.  Then  we  may  apply  Lemma  4.6  a  total  of  p 
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times  as  in  part  (4),  above,  to  conclude 

ff/.(0i(C.(ti,A,)))  =  =  5"  =  «r,,(Cj.p(32,72))  =  Aj))), 

as  required. 

(6).  Check  that  if  Aj))  =  then  <r,,(0i(Ci(t2,  Aj)))  =  tr,,(^,(h<)). 

First,  suppose  that  Ai))  =  is  a  distinguished  variable  v.  Then  from 

part  (4),  above,  we  know  that  Aj)))  =  r;  also,  by  Lemma  4.7,  part  1,  we  have 

as  reqtiired. 

On  the  other  hand,  suppose  o'j,(Ci(ti,  Aj))  =  <rjj(h*)  is  a  nondistinguished  variable,  say 
.4*.  Let  ^(Ci(ti,Ai))  =  2j(si,7i).  If  Aj)))  =  o’jj(^(h'))  **  *  distmguished 

variable,  then  we  may  apply  the  argument  of  the  previous  paragraph  to  finish.  So  sup¬ 
pose  0'i,(^(C,(ti,  Ai)))  =  o^/j(^(h‘))  is  a  nondistinguished  variable  5"*.  Then  si  >  M  by 
Lemma  4.11,  and  we  can  conclude  as  in  part  (5),  above,  that 

Ai)))  =  <r,,(C,.p(3i  -  p,7i))  =  =  tr,,(kj.p)  = 


by  an  application  of  Lemma  4.8  and  Lemma  4.7. 
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